⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, and smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅,支持关键词精准筛选。AI 智能筛选新闻 + AI 翻译 + AI 分析简报直推手机,也支持接入 MCP 架构,赋能 AI 自然语言对话分析、情感洞察与趋势预测等。支持 Docker ,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/bark/slack 等渠道智能推送。
Python aibarkdata-analysisdockerhot-newsllmmailmcpmcp-servernewsntfypythonrsstrending-topicswechatwework
v5.2.0: 新增 ai 翻译等功能,若干bug修复
sansan0246c8d09a41
diff --git a/.github/ISSUE_TEMPLATE/01-bug-report.yml b/.github/ISSUE_TEMPLATE/01-bug-report.ymlindex c15d771fef..ac26618ddd 100644--- a/.github/ISSUE_TEMPLATE/01-bug-report.yml+++ b/.github/ISSUE_TEMPLATE/01-bug-report.yml@@ -8,8 +8,30 @@ body:- type: markdownattributes:value: |+ ### ⚠️ 提交前必读+ **请确保你正在使用 TrendRadar 的最新版本。**+ 很多问题在最新代码中可能已经修复。如果你使用的是旧版本,我将无法处理,请先更新后再试。+**简单的描述 + 关键截图** 是最有效的沟通方式。+ - type: input+ id: version+ attributes:+ label: 📦 TrendRadar 版本+ description: 请务必提供。(如:v5.2.0 或 git commit id)+ placeholder: v5.2.0 或 commit hash+ validations:+ required: true++ - type: input+ id: mcp-version+ attributes:+ label: 🔌 MCP Server 版本 (可选)+ description: 如果你是通过 MCP 使用,请填写 MCP Server 的版本。+ placeholder: v3.1.6 (非 MCP 用户留空)+ validations:+ required: false+- type: dropdownid: bug-categoryattributes:diff --git a/.github/workflows/crawler.yml b/.github/workflows/crawler.ymlindex 1a3e804c82..ff6a912321 100644--- a/.github/workflows/crawler.yml+++ b/.github/workflows/crawler.yml@@ -154,7 +154,7 @@ jobs:# 通用Webhook配置GENERIC_WEBHOOK_URL: ${{ secrets.GENERIC_WEBHOOK_URL }}GENERIC_WEBHOOK_TEMPLATE: ${{ secrets.GENERIC_WEBHOOK_TEMPLATE }}- # AI 分析配置+ # AI 配置(ai_analysis 和 ai_translation 共享模型配置)AI_ANALYSIS_ENABLED: ${{ secrets.AI_ANALYSIS_ENABLED }}AI_API_KEY: ${{ secrets.AI_API_KEY }}AI_PROVIDER: ${{ secrets.AI_PROVIDER }}diff --git a/README-EN.md b/README-EN.mdindex 73ce500438..8331ae3b60 100644--- a/README-EN.md+++ b/README-EN.md@@ -13,9 +13,10 @@ Deploy in <strong>30 seconds</strong> — Say goodbye to endless scrolling, only[](https://github.com/sansan0/TrendRadar/stargazers)[](https://github.com/sansan0/TrendRadar/network/members)[](LICENSE)-[](https://github.com/sansan0/TrendRadar)-[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://work.weixin.qq.com/)[](https://weixin.qq.com/)@@ -56,8 +57,8 @@ Deploy in <strong>30 seconds</strong> — Say goodbye to endless scrolling, only| | | ||:---:|:---:|:---:|| [🚀 **Quick Start**](#-quick-start) | [AI Analysis](#-ai-analysis) | [⚙️ **Configuration Guide**](#configuration-guide) |-| [Docker Deployment](#6-docker-deployment) | [MCP Clients](#-mcp-clients) | [Related Projects](#-related-projects) |-| | [❓ **FAQ & Support**](#-faq--support) | |+| [Docker Deployment](#6-docker-deployment) | [MCP Clients](#-mcp-clients) | [📝 **Changelog**](#-changelog) |+| [🎯 **Core Features**](#-core-features) | [☕ **Support Project**](#-support-project) | [📚 **Related Projects**](#-related-projects) |</div>@@ -102,7 +103,7 @@ After communication, the author indicated no concerns about server pressure, but> Thanks to **financial supporters**. Your generosity has transformed into snacks and drinks beside my keyboard, accompanying every iteration of this project>> **Return of "One-Yuan Appreciation"**:-> With the release of v5.0.0, the project enters a new phase. To support growing API costs and caffeine consumption, the "One-Yuan Appreciation" channel is now reopened. Every bit of your kindness translates into Tokens and motivation in the code world. 🚀 [Support Now](#-faq--support)+> With the release of v5.0.0, the project enters a new phase. To support growing API costs and caffeine consumption, the "One-Yuan Appreciation" channel is now reopened. Every bit of your kindness translates into Tokens and motivation in the code world. 🚀 [Support Now](#-support-project)| Supporter | Amount (CNY) | Date | Note || :-------: | :----------: | :--: | :--: |@@ -148,11 +149,78 @@ After communication, the author indicated no concerns about server pressure, but<br>+## ☕ Support Project++> If this project is helpful to you, you can choose the following ways to support:+> 1. **Public Welfare**: Search for **Tencent Charity** on WeChat and donate to **Education Support** related projects as you wish.+>+> 2. **Sponsor the Developer**: Your sponsorship will be used to replenish caffeine for the carbon-based organism and Token consumption for the silicon-based organism.+++- **GitHub Issues**: Suitable for targeted answers. Please provide complete info when asking (screenshots, error logs, system environment, etc.).+- **Official Account**: Suitable for quick consultation. Suggest priority to communicate in public comment area of related articles. If private message, please use polite language 😉+- **Contact**: [email protected]+++| Official Account | WeChat Appreciation | Alipay Appreciation |+|:---:|:---:|:---:|+| <img src="_image/weixin.png" width="300" title="Silicon Tea Room"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F2ae0a88d98079f7e876c2b4dc85233c6-9e8025.JPG" width="300" title="WeChat Pay"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F1ed4f20ab8e35be51f8e84c94e6e239b4-fe4947.JPG" width="300" title="Alipay"/> |++<br>+## 📝 Changelog>**📌 Check Latest Updates**: **[Original Repository Changelog](https://github.com/sansan0/TrendRadar?tab=readme-ov-file#-changelog)**:- **Tip**: Check [Changelog] to understand specific [Features]+### 2026/01/17 - v5.2.0++> See config.yaml for details++**🌐 AI Translation**++- **Multi-language Translation**: Translate push content to any language+- **Batch Translation**: Smart batch processing to reduce API calls+- **Custom Prompts**: Customize translation style++**🔧 Configuration Optimization**++- **Standalone AI Model Config**: Analysis and translation share model config+- **Unified Region Switches**: Unified management of push region display+- **Custom Region Order**: Customize display order of each region++**✨ AI Analysis Enhancement**++- **AI Analysis Embedded in HTML**: Analysis results directly embedded in HTML reports, used by email notifications+- **Rich Style AI Section**: Gradient blue card layout, clearly separating analysis dimensions+- **Ranking Timeline Support**: AI can access precise ranking at each crawl time point+- **Section Reorganization (7→4)**: Consolidated into Core Trends, Sentiment & Controversy, Signals & Anomalies, Outlook & Strategy++**🔧 Multi-Model Adaptation**++- **Universal Parameter Passthrough**: Pass any advanced parameters to API+- **Gemini Adaptation**: Native parameter support with relaxed safety settings++**🐛 Bug Fixes**++- Fixed various known issues, improved system stability+++### 2026/01/10 - mcp-v3.0.0~v3.1.5++- **Breaking Change**: All tool return values unified to `{success, summary, data, error}` structure+- **Async Consistency**: All 21 tool functions wrapped with `asyncio.to_thread()` for sync calls+- **MCP Resources**: Added 4 resources (platforms, rss-feeds, available-dates, keywords)+- **RSS Enhancement**: `get_latest_rss` supports multi-day queries (days param), cross-date URL deduplication+- **Regex Matching Fix**: `get_trending_topics` supports `/pattern/` regex syntax and `display_name`+- **Cache Optimization**: Added `make_cache_key()` function with param sorting + MD5 hash for consistency+- **New check_version Tool**: Check TrendRadar and MCP Server version updates simultaneously+++<details>+<summary>👉 Click to expand: <strong>Historical Updates</strong></summary>++### 2026/01/10 - v5.0.0> **Dev Anecdote**:@@ -177,20 +245,6 @@ This update refactors the push message structure into five distinct core section- **Custom Prompts**: Customize AI analysis role and output format via `config/ai_analysis_prompt.txt`- **Multi-dimensional Analysis**: AI can analyze ranking changes, trending duration, cross-platform performance, trend prediction-### 2026/01/10 - mcp-v3.0.0~v3.1.5--- **Breaking Change**: All tool return values unified to `{success, summary, data, error}` structure-- **Async Consistency**: All 21 tool functions wrapped with `asyncio.to_thread()` for sync calls-- **MCP Resources**: Added 4 resources (platforms, rss-feeds, available-dates, keywords)-- **RSS Enhancement**: `get_latest_rss` supports multi-day queries (days param), cross-date URL deduplication-- **Regex Matching Fix**: `get_trending_topics` supports `/pattern/` regex syntax and `display_name`-- **Cache Optimization**: Added `make_cache_key()` function with param sorting + MD5 hash for consistency-- **New check_version Tool**: Check TrendRadar and MCP Server version updates simultaneously---<details>-<summary>👉 Click to expand: <strong>Historical Updates</strong></summary>-### 2026/01/02 - v4.7.0@@ -258,7 +312,7 @@ This update refactors the push message structure into five distinct core section**🎉 Major Update: Comprehensive Refactoring of Storage and Core Architecture**-- **Multi-Storage Backend Support**: Introduced a brand new storage module supporting local SQLite and remote cloud storage (S3-compatible protocols, Cloudflare R2 recommended for free tier), adaptable to GitHub Actions, Docker, and local environments.+- **Multi-Storage Backend Support**: Introduced a brand new storage module supporting local SQLite and remote cloud storage (S3-compatible protocols, e.g., Cloudflare R2), adaptable to GitHub Actions, Docker, and local environments.- **Database Structure Optimization**: Refactored SQLite database table structures to improve data efficiency and query performance.- **Enhanced Features**: Implemented date format standardization, data retention policies, timezone configuration support, and optimized time display. Fixed remote storage data persistence issues to ensure accurate data merging.- **Cleanup and Compatibility**: Removed most legacy compatibility code and unified data storage and retrieval methods.@@ -822,6 +876,31 @@ Supports **WeWork** (+ WeChat push solution), **Feishu**, **DingTalk**, **Telegr> 💡 For detailed configuration, see [Configuration Guide - Multi-Account Push Configuration](#10-multiple-account-push-configuration)+### **AI Multi-Language Translation** (v5.2.0 New)++Translate push content into any language, breaking language barriers — whether reading domestic trends or subscribing to international news via RSS, access everything in your native language++- **One-Click Translation**: Set `ai_translation.enabled: true` and target language in `config.yaml`+- **Multi-Language Support**: Supports English, Korean, Japanese, French, and any other language+- **Smart Batch Processing**: Automatically batches translations to reduce API calls and save costs+- **Custom Style**: Customize translation style and terminology via `ai_translation_prompt.txt`+- **Shared Model Config**: Shares the `ai` config section with AI analysis feature++```yaml+# config.yaml quick enable example+ai_translation:+ enabled: true+ language: "English" # Target translation language+```++> 💡 Translation shares model config with AI analysis — just configure `ai.api_key` once to use both features++**RSS Source References**: Here are some RSS feed collections for your reference+- [awesome-tech-rss](https://github.com/tuan3w/awesome-tech-rss) - Tech, startup, and programming blogs & media+- [awesome-rss-feeds](https://github.com/plenaryapp/awesome-rss-feeds) - Mainstream news media RSS from countries worldwide++> ⚠️ Some international media content may involve sensitive topics that AI models might refuse to translate. Please filter subscription sources based on your actual needs+### **Flexible Storage Architecture (v4.0.0 Major Update)****Multi-Backend Support**:@@ -846,7 +925,7 @@ Use AI models to deeply analyze push content, automatically generate trending in- **Flexible Push**: Choose original content only, AI analysis only, or both- **Custom Prompts**: Customize analysis perspective via `config/ai_analysis_prompt.txt`-> 💡 Detailed configuration tutorial: [AI Analysis Configuration](#12-ai-analysis-configuration)+> 💡 Detailed configuration tutorial: [Let AI help me analyze hot topics](#12-let-ai-help-me-analyze-hot-topics)### **Independent Display Section (v5.0.0 New)**@@ -868,15 +947,15 @@ AI conversational analysis system based on MCP (Model Context Protocol), enablin>> See [AI Analysis](#-ai-analysis) for details-### **Zero Technical Barrier Deployment**+### **Web Deployment**-One-click GitHub Fork to use, no programming required.+After running, the `index.html` generated in the root directory is the complete news report page.-> 30-second deployment: GitHub Pages (web browsing) supports one-click save as image for easy sharing+> **Deployment**: Click **Use this template** to create your repository, then deploy to Cloudflare Pages or GitHub Pages.>-> 1-minute deployment: WeWork (mobile notification)--**💡 Tip:** Want a **real-time updated** web version? After forking, go to your repo Settings → Pages and enable GitHub Pages. [Preview Effect](https://sansan0.github.io/TrendRadar/).+> **💡 Tip**: Enable GitHub Pages for an online URL. Go to Settings → Pages to enable. [Preview Effect](https://sansan0.github.io/TrendRadar/)+>+> ⚠️ The GitHub Actions auto-storage feature has been discontinued (this approach caused excessive load on GitHub servers, affecting platform stability).### **Reduce APP Dependencies**@@ -887,9 +966,9 @@ Transform from "algorithm recommendation captivity" to "actively getting the inf**Typical Scenarios:** Stock investment monitoring, brand sentiment tracking, industry trend watching, lifestyle news gathering-| Github Pages Effect (Mobile Adapted, Email Push) | Feishu Push Effect |-|:---:|:---:|-|  |  |+| Web Effect (Email Push) | Feishu Push Effect | AI Analysis Push Effect |+|:---:|:---:|:---:|+|  |  |  |<br>@@ -910,7 +989,7 @@ Transform from "algorithm recommendation captivity" to "actively getting the inf#### 🅱️ Option B: GitHub Actions Deployment (This Chapter ⬇️)* **Features**: Data is stored in **Remote Cloud Storage** (no longer written to Git repo)-* **Recommended**: Configure cloud storage service (Cloudflare R2 free tier is sufficient, Alibaba Cloud OSS, Tencent Cloud COS, etc.)+* **Storage**: Configure cloud storage service (e.g. Cloudflare R2, Alibaba Cloud OSS, Tencent Cloud COS, etc.)* **Note**: Requires periodic check-in renewal (every 7 days)### 1️⃣ Step 1: Get project code@@ -997,9 +1076,11 @@ Transform from "algorithm recommendation captivity" to "actively getting the inf</details><details>-<summary> <strong>👉 Click to expand: Feishu Bot</strong> (Most user-friendly message display)</summary>+<summary> <strong>👉 Click to expand: Feishu Bot</strong> (Message display is relatively friendly)</summary><br>+**Note**: If **AI Analysis** is enabled, Feishu push notifications may occasionally (approx. 5% probability) experience a few minutes of delay. This is likely due to the platform's internal compliance auditing for AI-generated content.+**GitHub Secret Configuration (⚠️ Name must match exactly):**- **Name**: `FEISHU_WEBHOOK_URL` (Please copy and paste this name, do not type manually)- **Secret (Value)**: Your Feishu bot Webhook address (link starts with https://www.feishu.cn/flow/api/trigger-webhook/********)@@ -1736,7 +1817,7 @@ Templates support two placeholders:- Need quick answers to problems?- Have great ideas to share?- 👉 Follow the WeChat Official Account「**[硅基茶水间](#-faq--support)**」, your likes and comments are the motivation for continuous updates.+ 👉 Follow the WeChat Official Account「**[Silicon Tea Room](#-support-project)**」, your likes and comments are the motivation for continuous updates.<br>@@ -2179,10 +2260,10 @@ BYD</details>-### 3. Push Mode Details+### 3. Which push mode should I choose?<details>-<summary>👉 Click to expand: <strong>Three Push Modes Detailed Comparison</strong></summary>+<summary>👉 Click to expand: <strong>Detailed Comparison of 3 Modes</strong></summary><br>**Configuration Location:** `report.mode` in `config/config.yaml`@@ -2192,8 +2273,6 @@ report:mode: "daily" # Options: "daily" | "incremental" | "current"```-**Docker Environment Variable:** `REPORT_MODE=incremental`-#### Detailed Comparison Table| Mode | Target Users | Push Timing | Display Content | Typical Use Case |@@ -2241,10 +2320,10 @@ Assume you monitor "Apple" keyword, execute once per hour:</details>-### 4. Advanced Configuration - Hotspot Weight Adjustment+### 4. How to adjust hotness algorithm?<details>-<summary>👉 Click to expand: <strong>Hotspot Weight Adjustment</strong></summary>+<summary>👉 Click to expand: <strong>Customize Hotspot Weights</strong></summary><br>**Configuration Location:** `advanced.weight` section in `config/config.yaml`@@ -2290,10 +2369,10 @@ Core idea: Users pursuing speed and timeliness increase ranking weight, users pu</details>-### 5. Push Format Reference+### 5. What will the messages look like?<details>-<summary>👉 Click to expand: <strong>Push Format Explanation</strong></summary>+<summary>👉 Click to expand: <strong>Message Style Preview</strong></summary><br>#### Push Example@@ -2422,16 +2501,12 @@ current directory/| Environment Variable | Corresponding Config | Example Value | Description ||---------------------|---------------------|---------------|-------------|- | `ENABLE_CRAWLER` | `advanced.crawler.enabled` | `true` / `false` | Enable crawler |- | `ENABLE_NOTIFICATION` | `notification.enabled` | `true` / `false` | Enable notification |- | `REPORT_MODE` | `report.mode` | `daily` / `incremental` / `current`| Report mode |- | `DISPLAY_MODE` | `report.display_mode` | `keyword` / `platform` | Display mode || `ENABLE_WEBSERVER` | - | `true` / `false` | Auto-start web server || `WEBSERVER_PORT` | - | `8080` | Web server port || `FEISHU_WEBHOOK_URL` | `notification.channels.feishu.webhook_url` | `https://...` | Feishu Webhook (multi-account use `;` separator) || `AI_ANALYSIS_ENABLED` | `ai_analysis.enabled` | `true` / `false` | Enable AI analysis (v5.0.0 new) |- | `AI_API_KEY` | `ai_analysis.api_key` | `sk-xxx...` | AI API Key (v5.0.0 new) |- | `AI_PROVIDER` | `ai_analysis.provider` | `deepseek` / `openai` / `gemini` | AI provider (v5.0.0 new) |+ | `AI_API_KEY` | `ai.api_key` | `sk-xxx...` | AI API Key (shared by ai_analysis and ai_translation) |+ | `AI_PROVIDER` | `ai.provider` | `deepseek` / `openai` / `gemini` | AI provider (v5.0.0 new) || `S3_*` | `storage.remote.*` | - | Remote storage config (5 params) |**Config Priority**: Environment Variables > config.yaml@@ -2757,10 +2832,10 @@ After MCP service starts, configure based on your client:</details>-### 7. Report Configuration+### 7. How is the push content displayed?<details>-<summary>👉 Click to expand: <strong>Report-Related Parameter Configuration</strong></summary>+<summary>👉 Click to expand: <strong>Customize Push Style and Content</strong></summary><br>**Configuration Location:** `report` section in `config/config.yaml`@@ -2811,11 +2886,6 @@ Controls how news is grouped in push messages and HTML reports:2. [Trump] Trump announces major policy #2 - 09:15 (3 times)```-**Docker Environment Variable:**-```bash-DISPLAY_MODE=platform-```-#### Content Order Configuration (v3.5.0 New)Controls display order of two content sections in push messages and HTML reports:@@ -2853,12 +2923,14 @@ MAX_NEWS_PER_KEYWORD=10Provides full trending list display for specified platforms, unaffected by `frequency_words.txt` keyword filtering.-**Configuration Location:** `notification.standalone_display` section in `config/config.yaml`+**Configuration Location:** `display` section in `config/config.yaml````yaml-notification:- standalone_display:- enabled: false # Enable or not+display:+ regions:+ standalone: true # Enable independent display section++ standalone:platforms: ["zhihu", "weibo"] # Trending platform ID listrss_feeds: ["hacker-news"] # RSS feed ID listmax_items: 20 # Max display count per source (0=unlimited)@@ -2884,10 +2956,10 @@ Hacker News (5 items):</details>-### 8. Push Window Configuration+### 8. When will I receive pushes?<details>-<summary>👉 Click to expand: <strong>Push Time Window Control Details</strong></summary>+<summary>👉 Click to expand: <strong>Set Push Time Window</strong></summary><br>**Configuration Location:** `notification.push_window` section in `config/config.yaml`@@ -2960,10 +3032,10 @@ notification:</details>-### 9. Execution Frequency Configuration+### 9. How often does it run?<details>-<summary>👉 Click to expand: <strong>Automatic Execution Frequency Settings</strong></summary>+<summary>👉 Click to expand: <strong>Set Auto-Run Frequency</strong></summary><br>**Configuration Location:** `schedule` section in `.github/workflows/crawler.yml`@@ -2974,618 +3046,212 @@ on:- cron: "0 * * * *" # Run every hour```-#### What is a Cron Expression?+#### How to change the schedule?-Cron is a time-based job scheduler format, consisting of 5 parts: `minute hour day month weekday`--```-┌───────────── minute (0-59)-│ ┌───────────── hour (0-23)-│ │ ┌───────────── day (1-31)-│ │ │ ┌───────────── month (1-12)-│ │ │ │ ┌───────────── weekday (0-6, 0=Sunday)-│ │ │ │ │-* * * * *-```+GitHub Actions uses a time format called "Cron". You don't need to understand it deeply; just copy and replace the code below.-#### Common Configuration Examples+**Configuration Location**: `schedule` section in `.github/workflows/crawler.yml`-| Desired Effect | Cron Expression | Description |-|---------------|----------------|-------------|-| Every hour | `0 * * * *` | Run at minute 0 of every hour (default) |-| Every 30 minutes | `*/30 * * * *` | Run every 30 minutes |-| Daily at 8 AM | `0 0 * * *` | UTC 0:00 = Beijing 8:00 AM |-| Working hours | `*/30 0-14 * * *` | Beijing 8:00-22:00, every 30 minutes |-| 3 times daily | `0 0,6,12 * * *` | Beijing 8:00, 14:00, 20:00 |+| I want... | Copy this line | Note |+|-----------|----------------|------|+| **Every Hour** | `- cron: "0 * * * *"` | **Default**, runs at minute 0 |+| **Every 30 Mins** | `- cron: "*/30 * * * *"` | Runs every 30 minutes |+| **Daily at 8 AM** | `- cron: "0 0 * * *"` | ⚠️ `0` because UTC 0:00 = Beijing 8:00 AM |+| **Work Hours (30m)** | `- cron: "*/30 0-14 * * *"` | Beijing 8:00 - 22:00 |+| **3 Times Daily** | `- cron: "0 0,6,12 * * *"` | Beijing 8:00, 14:00, 20:00 |-#### Important Notes+#### ⚠️ Two Important Notes-> ⚠️ **Time Zone Note**: GitHub Actions uses **UTC time**, Beijing time needs to **subtract 8 hours**-> - Want Beijing 8:00 AM run → Set UTC 0:00-> - Want Beijing 8:00 PM run → Set UTC 12:00+1. **Time Zone**: GitHub servers use **UTC time**.+ - **Math**: Your desired Beijing time **minus 8 hours** = value to fill.+ - *Example: For Beijing 20:00, fill in 12:00.*-> ⚠️ **Frequency Limit**: GitHub has a limit on Actions execution count per account-> - **Recommendation**: Don't set intervals shorter than 30 minutes-> - **Reason**: Too frequent may be considered abuse, facing account ban risk-> - **Reality**: GitHub Actions execution time has inherent deviation, setting too precise is meaningless+2. **Don't run too often**: Suggest intervals no shorter than 30 minutes.+ - GitHub free resources are limited; running too frequently might get flagged.+ - Actions have startup delays, so precise timing isn't guaranteed anyway.-#### Modification Method+#### Step-by-Step Guide-1. Open your forked repository-2. Find `.github/workflows/crawler.yml` file-3. Click edit (pencil icon)-4. Modify the expression in `cron: "0 * * * *"`-5. Click "Commit changes" to save+1. In your GitHub repository, find `.github/workflows/crawler.yml`.+2. Click the ✏️ (Edit) button top right.+3. Find the line `cron: "..."` and replace the content inside quotes with the code above.+4. Click the green **Commit changes** button to save.</details>-### 10. Multiple Account Push Configuration+### 10. Push to multiple groups/devices<details>-<summary>👉 Click to expand: <strong>Multiple Account Push Configuration Guide</strong></summary>+<summary>👉 Click to expand: <strong>Send to Multiple Recipients</strong></summary><br>**Configuration Location:** `notification` section in `config/config.yaml`-> ### ⚠️ **Security Warning**-> **GitHub Fork Users: DO NOT configure push information in `config.yaml`!**+> ### ⚠️ **Security First**+> **DO NOT write passwords/Tokens directly in `config.yaml`!**+> If you upload a file containing passwords to GitHub, the whole world can see it.>-> - **Risk**: `config.yaml` will be committed to public Git repositories. Configuring push information (Webhook URLs, Tokens, etc.) will expose sensitive data-> - **Recommended Methods**:-> - **GitHub Actions Users** → Use GitHub Secrets environment variables-> - **Docker Users** → Use [`.env` file configuration](#6-docker-deployment) (`.env` is in `.gitignore` and won't be committed)-> - **Local Development Users**: Can configure in `config.yaml` (ensure it won't be pushed to public repositories)--#### Multi-Account Push Notes (v3.5.0 New Feature)--- **Multi-Account Configuration Support**: All push channels (Feishu, DingTalk, WeWork, Telegram, ntfy, Bark, Slack) support configuring multiple accounts-- **Configuration Method**: Use English semicolon `;` to separate multiple account values-- **Example**: Set `FEISHU_WEBHOOK_URL` Secret value to `https://webhook1;https://webhook2`-- **Paired Configuration**: Telegram and ntfy require paired parameter quantities to match (e.g., token and chat_id both have 2 values)-- **Quantity Limit**: Default maximum 3 accounts per channel, exceeded values will be truncated--**Multi-Account Configuration Examples**:--| Name | Secret (Value) Example |-|------|------------------------|-| `FEISHU_WEBHOOK_URL` | `https://webhook1;https://webhook2;https://webhook3` |-| `TELEGRAM_BOT_TOKEN` | `token1;token2` |-| `TELEGRAM_CHAT_ID` | `chatid1;chatid2` |-| `NTFY_TOPIC` | `topic1;topic2` |-| `NTFY_TOKEN` | `;token2` (leave empty for first if no token needed) |--#### Supported Channels--| Channel | Configuration Item | Pairing Required | Description |-|---------|-------------------|-----------------|-------------|-| **Feishu** | `feishu_url` | No | Multiple webhook URLs |-| **DingTalk** | `dingtalk_url` | No | Multiple webhook URLs |-| **WeWork** | `wework_url` | No | Multiple webhook URLs |-| **Telegram** | `telegram_bot_token` + `telegram_chat_id` | ✅ Yes | Token and chat_id quantities must match |-| **ntfy** | `ntfy_topic` + `ntfy_token` | ✅ Yes | Topic and token quantities must match (token optional) |-| **Bark** | `bark_url` | No | Multiple push URLs |-| **Slack** | `slack_webhook_url` | No | Multiple webhook URLs |-| **Email** | `email_to` | - | Already supports multiple recipients (comma-separated), no modification needed |--#### Recommended Method 1: GitHub Actions Environment Variables--**Configuration Location**: GitHub Repo → Settings → Secrets and variables → Actions → Repository secrets--**Basic Configuration Example**:-```bash-# Multi-account quantity limit-MAX_ACCOUNTS_PER_CHANNEL=3+> **Correct Method**:+> - **GitHub Actions Users**: Add in Settings -> Secrets+> - **Docker Users**: Write in `.env` file (this file won't be uploaded)-# Feishu multi-account (3 groups)-FEISHU_WEBHOOK_URL=https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy;https://hook3.feishu.cn/zzz+#### How to push to multiple places?-# DingTalk multi-account (2 groups)-DINGTALK_WEBHOOK_URL=https://oapi.dingtalk.com/xxx;https://oapi.dingtalk.com/yyy+Simple, just separate multiple addresses with a semicolon `;`.-# WeWork multi-account (2 groups)-WEWORK_WEBHOOK_URL=https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx;https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yyy+**Example**:+Suppose you have two Feishu groups:+- Group 1: `https://.../webhook/aaa`+- Group 2: `https://.../webhook/bbb`-# Bark multi-account (2 devices)-BARK_URL=https://api.day.app/key1;https://api.day.app/key2+Config value:+`https://.../webhook/aaa;https://.../webhook/bbb`-# Slack multi-account (2 channels)-SLACK_WEBHOOK_URL=https://hooks.slack.com/xxx;https://hooks.slack.com/yyy-```+#### Supported Platforms-**Paired Configuration Examples (Telegram and ntfy)**:+| Platform | Method | Note |+|----------|--------|------|+| **Feishu/DingTalk/WeWork** | Separate URLs with `;` | Just chain them up |+| **Bark (iOS)** | Separate URLs with `;` | Push to multiple iPhones |+| **Telegram** | Separate Tokens and ChatIDs with `;` | ⚠️ **Order must match**: <br>Token1 ↔ ChatID1<br>Token2 ↔ ChatID2 |+| **ntfy** | Separate Topics and Tokens with `;` | If a topic needs no token, leave empty:<br>`token1;;token3` (middle is empty) |-<details>-<summary><strong>Telegram Paired Configuration</strong></summary>+#### Common Config Examples (GitHub Secrets / .env)```bash-# ✅ Correct: 2 tokens correspond to 2 chat_ids-TELEGRAM_BOT_TOKEN=123456:AAA-BBB;789012:CCC-DDD-TELEGRAM_CHAT_ID=-100111;-100222+# Send to 3 Feishu groups+FEISHU_WEBHOOK_URL=https://hook1...;https://hook2...;https://hook3...-# ❌ Incorrect: quantities don't match, push will be skipped-TELEGRAM_BOT_TOKEN=token1;token2;token3-TELEGRAM_CHAT_ID=id1;id2-```--**Note**: The quantities of `token` and `chat_id` must match exactly, otherwise the channel push will be skipped.--</details>--<details>-<summary><strong>ntfy Paired Configuration</strong></summary>--```bash-# ✅ Correct: 3 topics, only the 2nd needs a token-NTFY_TOPIC=topic1;topic2;topic3-NTFY_TOKEN=;token_for_topic2;+# Send to 2 DingTalk groups+DINGTALK_WEBHOOK_URL=https://oapi...;https://oapi...-# ✅ Correct: 2 topics both need tokens-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2--# ❌ Incorrect: topic and token quantities don't match-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2;token3+# Send to 2 Telegram users (Match one-to-one)+TELEGRAM_BOT_TOKEN=tokenA;tokenB+TELEGRAM_CHAT_ID=userA;userB```-**Notes**:-- If a topic doesn't need a token, leave it empty at the corresponding position (between two semicolons)-- The quantities of `topic` and `token` must match+> **Tip**: Default limit is 3 accounts per platform to prevent abuse. Adjust `MAX_ACCOUNTS_PER_CHANNEL` if needed.</details>------#### Recommended Method 2: Docker Environment Variables (.env)--**Configuration Location**: `docker/.env` file in project root directory--**Basic Configuration Example**:-```bash-# Multi-account quantity limit-MAX_ACCOUNTS_PER_CHANNEL=3--# Feishu multi-account (3 groups)-FEISHU_WEBHOOK_URL=https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy;https://hook3.feishu.cn/zzz--# DingTalk multi-account (2 groups)-DINGTALK_WEBHOOK_URL=https://oapi.dingtalk.com/xxx;https://oapi.dingtalk.com/yyy--# WeWork multi-account (2 groups)-WEWORK_WEBHOOK_URL=https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx;https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yyy--# Bark multi-account (2 devices)-BARK_URL=https://api.day.app/key1;https://api.day.app/key2--# Slack multi-account (2 channels)-SLACK_WEBHOOK_URL=https://hooks.slack.com/xxx;https://hooks.slack.com/yyy-```--**Paired Configuration Examples (Telegram and ntfy)**:--<details>-<summary><strong>Telegram Paired Configuration</strong></summary>--```bash-# ✅ Correct: 2 tokens correspond to 2 chat_ids-TELEGRAM_BOT_TOKEN=123456:AAA-BBB;789012:CCC-DDD-TELEGRAM_CHAT_ID=-100111;-100222--# ❌ Incorrect: quantities don't match, push will be skipped-TELEGRAM_BOT_TOKEN=token1;token2;token3-TELEGRAM_CHAT_ID=id1;id2-```--**Note**: The quantities of `token` and `chat_id` must match exactly, otherwise the channel push will be skipped.--</details>--<details>-<summary><strong>ntfy Paired Configuration</strong></summary>--```bash-# ✅ Correct: 3 topics, only the 2nd needs a token-NTFY_TOPIC=topic1;topic2;topic3-NTFY_TOKEN=;token_for_topic2;--# ✅ Correct: 2 topics both need tokens-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2--# ❌ Incorrect: topic and token quantities don't match-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2;token3-```--**Notes**:-- If a topic doesn't need a token, leave it empty at the corresponding position (between two semicolons)-- The quantities of `topic` and `token` must match--</details>-------#### Push Behavior Description--1. **Independent Push**: Each account sends independently, one failure doesn't affect other accounts-2. **Partial Success**: As long as one account sends successfully, the overall result is considered successful-3. **Log Differentiation**: Multi-account logs show "Account 1", "Account 2", etc.-4. **Batch Interval**: Multi-account increases total send time (each account independently calculates batch interval)-------#### FAQ--<details>-<summary><strong>Q1: What happens if more than 3 accounts are configured?</strong></summary>-<br>--The system will automatically truncate to the configured maximum quantity and output warning logs. You can adjust the limit via `max_accounts_per_channel`.--**⚠️ Special Note for GitHub Actions Users**:-- **Not recommended to configure too many accounts** (suggest no more than 3), which may cause:- - **Trigger GitHub Actions rate limits**: Frequent network requests may be identified as abnormal behavior- - **Affect Workflow execution time**: Multi-account pushing will extend runtime and may exceed free quotas- - **Potential account risks**: Excessive use of GitHub Actions resources may affect account status-- **Recommended Practices**:- - Use `max_accounts_per_channel: 3` or lower values- - Adjust Cron execution frequency (e.g., change from hourly to every 2-3 hours)- - Prioritize the most important push channels and avoid configuring too many accounts--</details>--<details>-<summary><strong>Q2: Will multi-accounts affect push speed?</strong></summary><br>-Yes. Each account sends independently, total time = number of accounts × single account send time. Recommend controlling the number of accounts.--</details>+### 11. Where is the data saved?-<details>-<summary><strong>Q3: How can local development users configure in config.yaml?</strong></summary>+<details id="storage-config">+<summary>👉 Click to expand: <strong>Choose Data Storage Location</strong></summary><br>-If you are doing local development and **will not push code to public repositories**, you can configure directly in `config/config.yaml`:--```yaml-notification:- enabled: true-- channels:- feishu:- webhook_url: "https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy"- telegram:- bot_token: "token1;token2"- chat_id: "id1;id2"+#### Where is the data saved?-advanced:- max_accounts_per_channel: 3-```+The system automatically selects the best location for you, so you usually don't need to worry about it:-**⚠️ Important Reminder**:-- Ensure `config/config.yaml` is in `.gitignore` (if you will commit code)-- Or only use in local development environment, **never commit to public repositories**+| Your Environment | Data Location | Description |+|------------------|---------------|-------------|+| **Docker / Local** | **Local Disk** | Saved in the `output/` folder within the project directory. |+| **GitHub Actions** | **Cloud Storage** | Since GitHub Actions environments are destroyed after running, cloud storage (e.g., Cloudflare R2) is required. |-</details>+#### How to configure cloud storage? (For GitHub Actions Users)-</details>+If you run on GitHub Actions, you need a "cloud drive" to save data. For example, Cloudflare R2 (free tier is generous).-<br>+**Add these 5 variables to GitHub Secrets:**-### 11. Storage Configuration (v4.0.0 New)+| Variable Name | Value |+|---------------|-------|+| `STORAGE_BACKEND` | `remote` |+| `S3_BUCKET_NAME` | Your bucket name |+| `S3_ACCESS_KEY_ID` | Your Access Key |+| `S3_SECRET_ACCESS_KEY` | Your Secret Key |+| `S3_ENDPOINT_URL` | Your R2 endpoint URL |-<details>-<summary>👉 Click to expand: <strong>Storage Configuration Guide</strong></summary>-<br>--#### Storage Backend Selection--TrendRadar v4.0.0 introduces **multi-backend storage architecture**, supporting automatic backend selection or manual specification:--| Configuration Value | Description | Applicable Scenarios |-|---------------------|-------------|---------------------|-| `auto` (default) | Auto-select backend: GitHub Actions→R2, other environments→Local | Most users (recommended) |-| `local` | Force use of local SQLite | Docker/local deployment |-| `r2` | Force use of Cloudflare R2 | Cloud storage required |--**Configuration Location**:-- GitHub Actions: Set `STORAGE_BACKEND` environment variable in GitHub Secrets-- Docker: Configure `STORAGE_BACKEND=local` in `.env` file-- Local: Add `STORAGE_BACKEND` in environment variables or use auto mode-------#### Database Structure Optimization (v4.0.0)--v4.0.0 made significant optimizations to database structure, removing redundant fields and improving data normalization:--##### 1. Removed Redundant Fields--Removed the following redundant fields from `news` table:--| Field Name | Removal Reason | Alternative |-|------------|----------------|------------|-| `source_name` | Duplicate with platform name | Get via `platforms` table JOIN query |-| `crawl_date` | Duplicate with file path date | Infer from file path timestamp |--**Migration Notes**: Old databases are incompatible, see [Breaking Changes](#breaking-changes-v400) section--##### 2. New Platforms Table--Added `platforms` table for unified management of platform information:--```sql-CREATE TABLE IF NOT EXISTS platforms (- id TEXT PRIMARY KEY, -- Platform ID (immutable, e.g., 'zhihu', 'weibo')- name TEXT NOT NULL, -- Platform display name (mutable, e.g., 'Zhihu', 'Weibo')- enabled INTEGER DEFAULT 1 -- Whether enabled (1=enabled, 0=disabled)-);-```--**Design Advantages**:-- `id` field is immutable, maintains data consistency-- `name` field is mutable, supports internationalization and customization-- Historical data remains valid when modifying platform names--##### 3. Crawl Source Status Normalization--Replaced original comma-separated string storage `successful_sources` field with normalized `crawl_source_status` table:--```sql-CREATE TABLE IF NOT EXISTS crawl_source_status (- id INTEGER PRIMARY KEY AUTOINCREMENT,- file_path TEXT NOT NULL, -- File path (e.g., 'output/2025-12-09/news.db')- platform_id TEXT NOT NULL, -- Platform ID (foreign key to platforms.id)- success INTEGER NOT NULL, -- Whether crawl succeeded (1=success, 0=failed)- crawl_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,- FOREIGN KEY (platform_id) REFERENCES platforms(id)-);-```--**Design Advantages**:-- Supports efficient SQL queries (e.g., calculate success rate by platform)-- Easy statistics and analysis (no string splitting required)-- Normalized structure, avoids data redundancy--##### 4. File Path Format Standardization--**Old Format**: `output/2025年12月09日/news_14-30.txt`-**New Format**: `output/2025-12-09/news.db`--**Changes**:-- Date format: Chinese format → ISO 8601 standard format-- Filename: Multiple time-stamped TXT files → single SQLite database file-- Extension: `.txt` → `.db`--**Advantages**:-- Cross-platform compatibility (avoids Chinese path issues)-- Easier programmatic parsing-- International standard, better maintainability-----+> 💡 **Tutorial**: How to apply for R2? See [Quick Start - Remote Storage Configuration](#-quick-start)-#### Remote Cloud Storage Configuration+#### How long is data kept?-When using remote cloud storage (required for GitHub Actions environment), configure the following environment variables:+By default, we never delete your data. If you want to save space, you can enable "Auto Cleanup".-| Environment Variable | Description | Required | Example Value |-|----------------------|-------------|----------|--------------|-| `S3_BUCKET_NAME` | Bucket name | ✅ Yes | `trendradar-data` |-| `S3_ACCESS_KEY_ID` | Access key ID | ✅ Yes | `abc123...` |-| `S3_SECRET_ACCESS_KEY` | Access key | ✅ Yes | `xyz789...` |-| `S3_ENDPOINT_URL` | S3 API endpoint | ✅ Yes | `https://<account-id>.r2.cloudflarestorage.com` |-| `S3_REGION` | Region (optional) | ❌ No | `auto` |+**Config Location**: `config/config.yaml`-**Configuration Method**:-- GitHub Actions: Configure in GitHub Secrets (see [Quick Start - Remote Storage Configuration](#2-setup-github-secrets-required--optional-platforms))-- Docker/Local: Configure in `.env` file (remote storage is optional)-------#### Data Cleanup Strategy--v4.0.0 added automatic data cleanup feature, supporting scheduled cleanup of old data:--**Configuration Items**: `LOCAL_RETENTION_DAYS` and `REMOTE_RETENTION_DAYS`--| Configuration Value | Description |-|---------------------|-------------|-| `0` (default) | Disable cleanup, keep all data |-| Positive integer (e.g., `30`) | Only keep recent N days of data, auto-delete old data |--**Configuration Method**:-```bash-# GitHub Actions: Configure in GitHub Secrets-LOCAL_RETENTION_DAYS=30-REMOTE_RETENTION_DAYS=30--# Docker: Configure in .env file-LOCAL_RETENTION_DAYS=30-REMOTE_RETENTION_DAYS=30--# Local: Add to environment variables-export LOCAL_RETENTION_DAYS=30+```yaml+storage:+ local:+ retention_days: 30 # Keep local data for 30 days (0 = forever)+ remote:+ retention_days: 30 # Keep cloud data for 30 days```-**Cleanup Rules**:-- Cleanup executes during each crawl task-- Local: Deletes database files older than N days (e.g., `output/news/2025-11-10.db`, `output/rss/2025-11-10.db`)-- Remote: Deletes cloud objects older than N days (e.g., `news/2025-11-10.db`, `rss/2025-11-10.db`)-------#### Timezone Configuration+#### Push time is wrong? (Timezone Settings)-v4.0.0 added timezone configuration support, using IANA standard time zone names:+If you are overseas or find the push time doesn't match your local time, you can change the timezone.-**Configuration Item**: `TIMEZONE`+**Config Location**: `config/config.yaml`-| Configuration Value | Description | Example |-|---------------------|-------------|---------|-| Not set (default) | Use UTC+0 | - |-| IANA time zone name | Specify time zone | `Asia/Shanghai`, `America/New_York`, `Europe/London` |--**Configuration Method**:-```bash-# GitHub Actions: Configure in GitHub Secrets-TIMEZONE=Asia/Shanghai--# Docker: Configure in .env file-TIMEZONE=Asia/Shanghai--# Local: Add to environment variables-export TIMEZONE=Asia/Shanghai+```yaml+app:+ timezone: "Asia/Shanghai" # Default is China Standard Time```--**Common IANA Time Zones**:-- China: `Asia/Shanghai`-- United States East: `America/New_York`-- United States West: `America/Los_Angeles`-- United Kingdom: `Europe/London`-- Japan: `Asia/Tokyo`-------#### Breaking Changes (v4.0.0)--**⚠️ Important Notice**: v4.0.0 made breaking changes to database structure, **old databases are incompatible**--**Impact**:-- Cannot directly read v3.x version data-- Need to re-crawl data to build new database-- **No automatic migration tool provided**--**Recommendations**:-1. **Fresh Start**: Recommended to start from scratch to accumulate data-2. **Keep Historical Data**: If need to preserve v3.x historical data, can rename old `output/` directory (e.g., `output_v3_backup/`) before running new version--**Data Format Comparison**:--| Item | v3.x | v4.0.0 |-|------|------|--------|-| File path format | `output/2025年12月09日/` | `output/2025-12-09/` |-| Data file | Multiple `news_HH-MM.txt` files | Single `news.db` file |-| Database fields | Contains `source_name`, `crawl_date` | Removed redundant fields |-| Platform management | No independent table | Added `platforms` table |-| Crawl status | Comma-separated string | Normalized `crawl_source_status` table |+- Example for Los Angeles: `America/Los_Angeles`+- Example for London: `Europe/London`</details>-### 12. AI Analysis Configuration+### 12. Let AI help me analyze hot topics<details id="ai-analysis-config">-<summary>👉 Click to expand: <strong>AI Analysis Push Configuration Guide</strong></summary>-<br>--#### Feature Overview--v5.0.0 adds AI analysis push feature, using AI models to deeply analyze push content and automatically generate trending insights report.+<summary>👉 Click to expand: <strong>Enable AI Smart Analysis</strong></summary>-**Analysis includes**:-- Trending topic overview-- Keyword popularity analysis-- Cross-platform correlation analysis-- Potential impact assessment-- Signals worth attention-- Summary and recommendations+#### What can AI do for me?-#### Configuration Location+After enabling this feature, AI acts as a professional analyst. When pushing a batch of news, it will:+1. **Auto-Read**: Read all matched trending news.+2. **Deep Think**: Analyze connections between seemingly isolated news items.+3. **Write Report**: Append a concise and profound "Insight Report" at the end of the push message.-**Config file**: `ai_analysis` section in `config/config.yaml`+**Includes**: Trending topic summary, public opinion direction, cross-platform correlation, potential impact assessment, etc.-```yaml-ai_analysis:- enabled: false # Enable AI analysis- provider: "deepseek" # AI provider- api_key: "" # API Key (recommend using AI_API_KEY env var)- model: "deepseek-chat" # Model name- base_url: "" # Custom API endpoint (optional)- timeout: 90 # Request timeout (seconds)- push_mode: "both" # Push mode- max_news_for_analysis: 50 # Max news items to analyze- include_rss: true # Include RSS content- prompt_file: "ai_analysis_prompt.txt" # Prompt config file-```--#### Supported AI Providers--| Provider | provider value | Default endpoint |-|----------|---------------|------------------|-| **DeepSeek** | `deepseek` | https://api.deepseek.com/v1/chat/completions |-| **OpenAI** | `openai` | https://api.openai.com/v1/chat/completions |-| **Google Gemini** | `gemini` | https://generativelanguage.googleapis.com/v1beta/openai/chat/completions |-| **Custom** | `custom` | Requires base_url |--> 💡 **Tip**: When using `custom` provider, `base_url` must be the complete API address (e.g., `https://api.example.com/v1/chat/completions`)--#### Push Mode Description--| Mode | Description |-|------|-------------|-| `only_analysis` | Push AI analysis result only, no original content |-| `both` | Push both (default), AI analysis appended after original content |--> 💡 **Tip**: If you don't need AI analysis, set `enabled` to `false` instead of using `push_mode`+#### How to enable AI Analysis?-#### Environment Variable Support+The simplest way is via environment variables (Recommended for GitHub Secrets or .env).-| Environment Variable | Description | Example |-|---------------------|-------------|---------|-| `AI_ANALYSIS_ENABLED` | Enable AI analysis | `true` / `false` |-| `AI_API_KEY` | AI API Key | `sk-xxx...` |-| `AI_PROVIDER` | AI provider | `deepseek` / `openai` / `gemini` / `custom` |-| `AI_MODEL` | Model name | `deepseek-chat` |-| `AI_BASE_URL` | Complete API address (required for custom) | `https://api.example.com/v1/chat/completions` |+**Required Configurations**:-#### Custom Prompts+| Variable Name | Value | Description |+|--------------|-------|-------------|+| `AI_ANALYSIS_ENABLED` | `true` | Enable switch |+| `AI_API_KEY` | `sk-xxxxxx` | Your API Key |+| `AI_PROVIDER` | `deepseek` | AI Provider (see table below) |+| `AI_MODEL` | `deepseek-chat` | Model Name |-Edit `config/ai_analysis_prompt.txt` to customize AI analysis role and output format.+**Supported AI Providers**:-**File structure**:-```-[system]-System prompt, define AI role and analysis principles-...+| Provider | AI_PROVIDER Value | Default Model (AI_MODEL) |+|----------|-------------------|------------------------|+| **DeepSeek** (Recommended) | `deepseek` | `deepseek-chat` |+| **OpenAI** | `openai` | `gpt-4o` |+| **Google Gemini** | `gemini` | `gemini-1.5-flash` |+| **Custom** (OneAPI) | `custom` | Requires `AI_BASE_URL` |-[user]-User prompt template, supports variable substitution-...-```+> 💡 **Tip**: DeepSeek offers excellent performance/price ratio, highly suitable for high-frequency news analysis.-**Available variables**:-- `{report_mode}` - Current report mode-- `{report_type}` - Report type description-- `{current_time}` - Current time-- `{news_count}` - Trending news count-- `{rss_count}` - RSS news count-- `{keywords}` - Matched keywords list-- `{platforms}` - Data source platforms list-- `{news_content}` - News content+#### Advanced: AI Translation-#### Quick Enable Example+If you subscribe to foreign RSS feeds (like Hacker News), AI can translate the content into your native language.-**Method 1: Config file**+**Configuration Location**: `config/config.yaml````yaml-ai_analysis:- enabled: true- provider: "deepseek"- api_key: "sk-your-api-key"- model: "deepseek-chat"- push_mode: "both"+ai_translation:+ enabled: true # Enable translation+ language: "Chinese" # Target language (Chinese, English, Japanese...)```-**Method 2: Environment variables (Recommended)**+#### Advanced: Customize AI "Persona"-```bash-# GitHub Actions: Add to Secrets-# Docker: Add to .env file-AI_ANALYSIS_ENABLED=true-AI_API_KEY=sk-your-api-key-AI_PROVIDER=deepseek-AI_MODEL=deepseek-chat-```+Think the AI sounds too official? You can modify its prompt to change its style (e.g., "Sarcastic Commentator", "Senior Investment Advisor").++- **File**: `config/ai_analysis_prompt.txt`+- **Method**: Edit with a text editor, tell AI what analysis style you want.</details>@@ -3623,7 +3289,7 @@ AI analysis **does not** query real-time online data directly, but analyzes **loCherry Studio provides GUI config interface, 5-minute quick deployment, complex parts are one-click install.-**Illustrated Deployment Tutorial**: Now updated to my WeChat Official Account (see [FAQ & Support](#-faq--support)), reply "mcp" to get+**Illustrated Deployment Tutorial**: Now updated to my WeChat Official Account (see [Support Project](#-support-project)), reply "mcp" to get**Detailed Deployment Tutorial**: [README-Cherry-Studio.md](README-Cherry-Studio.md)@@ -3639,7 +3305,7 @@ Cherry Studio provides GUI config interface, 5-minute quick deployment, complex> 💡 **Tip**: Actually not recommended to ask multiple questions at once. If your chosen AI model cannot even sequentially call as shown below, suggest switching models.-<img src="/_image/ai2.png" alt="MCP usage effect" width="600">+<img src="/_image/ai4.png" alt="MCP usage effect" width="600"><br>@@ -3950,25 +3616,6 @@ Any client supporting Model Context Protocol can connect to TrendRadar:<br>-## ☕ FAQ & Support--> If this project is helpful to you, you can choose the following ways to support:-> 1. **Public Welfare**: Search for **Tencent Charity** on WeChat and donate to **Education Support** related projects as you wish.->-> 2. **Sponsor the Developer**: Your sponsorship will be used to replenish caffeine for the carbon-based organism and Token consumption for the silicon-based organism.---- **GitHub Issues**: Suitable for targeted answers. Please provide complete info when asking (screenshots, error logs, system environment, etc.).-- **Official Account**: Suitable for quick consultation. Suggest priority to communicate in public comment area of related articles. If private message, please use polite language 😉-- **Contact**: [email protected]---| Official Account | WeChat Appreciation | Alipay Appreciation |-|:---:|:---:|:---:|-| <img src="_image/weixin.png" width="300" title="Silicon Tea Room"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F2ae0a88d98079f7e876c2b4dc85233c6-9e8025.JPG" width="300" title="WeChat Pay"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F1ed4f20ab8e35be51f8e84c94e6e239b4-fe4947.JPG" width="300" title="Alipay"/> |--<br>-## 📚 Related Projects> **4 Related Articles** (Chinese):diff --git a/README-MCP-FAQ-EN.md b/README-MCP-FAQ-EN.mdindex 63ebc21dda..dff1414c1c 100644--- a/README-MCP-FAQ-EN.md+++ b/README-MCP-FAQ-EN.md@@ -6,7 +6,7 @@# TrendRadar MCP Tool Usage Q&A-> AI Query Guide - How to Use News Trend Analysis Tools Through Natural Conversation (v3.1.5)+> AI Query Guide - How to Use News Trend Analysis Tools Through Natural Conversation (v3.1.6)---diff --git a/README-MCP-FAQ.md b/README-MCP-FAQ.mdindex 3cdbbbfb75..9eb67288d7 100644--- a/README-MCP-FAQ.md+++ b/README-MCP-FAQ.md@@ -6,7 +6,7 @@# TrendRadar MCP 工具使用问答-> AI 提问指南 - 如何通过自然对话使用新闻热点分析工具(v3.1.5)+> AI 提问指南 - 如何通过自然对话使用新闻热点分析工具(v3.1.6)---diff --git a/README.md b/README.mdindex 413c8847c4..6070feb5cf 100644--- a/README.md+++ b/README.md@@ -13,9 +13,10 @@[](https://github.com/sansan0/TrendRadar/stargazers)[](https://github.com/sansan0/TrendRadar/network/members)[](LICENSE)-[](https://github.com/sansan0/TrendRadar)-[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://github.com/sansan0/TrendRadar)+[](https://github.com/sansan0/TrendRadar)[](https://work.weixin.qq.com/)[](https://weixin.qq.com/)@@ -56,8 +57,8 @@| | | ||:---:|:---:|:---:|| [🚀 **快速开始**](#-快速开始) | [AI 智能分析](#-ai-智能分析) | [⚙️ **配置详解**](#配置详解) |-| [Docker部署](#6-docker-部署) | [MCP客户端](#-mcp-客户端) | [项目相关](#-项目相关) |-| | [❓ **答疑与交流**](#问题答疑与交流) | |+| [Docker部署](#6-docker-部署) | [MCP客户端](#-mcp-客户端) | [📝 **更新日志**](#-更新日志) |+| [🎯 **核心功能**](#-核心功能) | [☕ **支持项目**](#-支持项目) | [📚 **项目相关**](#-项目相关) |</div>@@ -102,7 +103,7 @@> 感谢**给予资金支持**的朋友们,你们的慷慨已化身为键盘旁的零食饮料,陪伴着项目的每一次迭代。>> **关于"一元点赞"的回归**:-> 随着 v5.0.0 版本的发布,项目迈入了一个新的阶段。为了支持日益增长的 API 成本和咖啡因消耗,"一元点赞"通道现已重新开启。你的每一份心意,都将转化为代码世界里的 Token 和动力。🚀 [前往支持](#问题答疑与交流)+> 随着 v5.0.0 版本的发布,项目迈入了一个新的阶段。为了支持日益增长的 API 成本和咖啡因消耗,"一元点赞"通道现已重新开启。你的每一份心意,都将转化为代码世界里的 Token 和动力。🚀 [前往支持](#-支持项目)| 点赞人 | 金额 | 日期 | 备注 || :-------------------------: | :----: | :----: | :-----------------------: |@@ -195,17 +196,84 @@<br>+## ☕ 支持项目++> 如果本项目对你有帮助,你可以选择以下方式支持:+> 1. **公益助学**:微信搜索**腾讯公益**,对里面的**助学**相关的项目随心捐。+>+> 2. **赞助开发者**:你的赞助将用于补充碳基生物的咖啡因和硅基生物的 Token 消耗。+++- **GitHub Issues**:适合针对性强的解答。提问时请提供完整信息(截图、错误日志、系统环境等)。+- **公众号交流**:适合快速咨询。建议优先在相关文章下的公共留言区交流,如私信,请文明礼貌用语😉+- **联系方式**:[email protected]+++|公众号关注 |微信点赞 | 支付宝点赞 |+|:---:|:---:|:---:|+| <img src="_image/weixin.png" width="300" title="硅基茶水间"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F2ae0a88d98079f7e876c2b4dc85233c6-9e8025.JPG" width="300" title="微信支付"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F1ed4f20ab8e35be51f8e84c94e6e239b4-fe4947.JPG" width="300" title="支付宝支付"/> |++<br>+## 📝 更新日志> **📌 查看最新更新**:**[原仓库更新日志](https://github.com/sansan0/TrendRadar?tab=readme-ov-file#-更新日志)** :- **提示**:建议查看【历史更新】,明确具体的【功能内容】+### 2026/01/17 - v5.2.0++> 主要见 config.yaml 描述++**🌐 AI 翻译功能**++- **多语言翻译**:支持将推送内容翻译为任意语言+- **批量翻译**:智能批量处理,减少 API 调用次数+- **自定义提示词**:支持自定义翻译风格++**🔧 配置架构优化**++- **AI 模型配置独立**:分析和翻译共享模型配置+- **区域开关统一**:统一管理推送区域显示+- **区域排序自定义**:支持自定义各区域的显示顺序++**✨ AI 分析增强**++- **AI 分析嵌入 HTML**:分析结果直接嵌入 HTML 报告,邮件通知直接使用+- **富样式 AI 区块**:渐变蓝色背景卡片式布局,清晰分隔各分析维度+- **排名时间线支持**:AI 可获取每条新闻在每个抓取时间点的精确排名+- **板块重组 (7→4)**:整合为核心热点态势、舆论风向争议、异动与弱信号、研判策略建议++**🔧 多模型适配**++- **通用参数透传**:支持向 API 透传任意高级参数+- **Gemini 适配**:原生参数支持,内置安全策略放宽++**🐛 Bug 修复**++- 修复若干已知问题,提升系统稳定性+++### 2026/01/10 - mcp-v3.0.0~v3.1.5++- **Breaking Change**:所有工具返回值统一为 `{success, summary, data, error}` 结构+- **异步一致性**:所有 21 个工具函数使用 `asyncio.to_thread()` 包装同步调用+- **MCP Resources**:新增 4 个资源(platforms、rss-feeds、available-dates、keywords)+- **RSS 增强**:`get_latest_rss` 支持多日查询(days 参数),跨日期 URL 去重+- **正则匹配修复**:`get_trending_topics` 支持 `/pattern/` 正则语法和 `display_name`+- **缓存优化**:新增 `make_cache_key()` 函数,参数排序+MD5 哈希确保一致性+- **新增 check_version 工具**:支持同时检查 TrendRadar 和 MCP Server 版本更新+++<details>+<summary>👉 点击展开:<strong>历史更新</strong></summary>++### 2026/01/10 - v5.0.0-> **开发小插曲**:+> **开发小插曲**:> 致敬那个陪伴我两年多、却在刚续费后反手弹出 `"This organization has been disabled"` 的某 C 厂模型-**✨ 推送内容“五大板块”重构**+**✨ 推送内容"五大板块"重构**本次更新对推送消息进行了区域化重构,现在推送内容清晰地划分为五大核心板块:@@ -242,21 +310,7 @@- **频率词配置增强**:新增 `[组别名]` 语法,支持 `#` 注释行,配置更清晰(感谢 [@songge8](https://github.com/sansan0/TrendRadar/issues/752) 提出的建议)- **环境变量支持**:AI 分析相关配置支持环境变量覆盖(`AI_API_KEY`、`AI_PROVIDER` 等)-> 💡 详细配置教程见 [AI 分析配置](#12-ai-分析配置)--### 2026/01/10 - mcp-v3.0.0~v3.1.5--- **Breaking Change**:所有工具返回值统一为 `{success, summary, data, error}` 结构-- **异步一致性**:所有 21 个工具函数使用 `asyncio.to_thread()` 包装同步调用-- **MCP Resources**:新增 4 个资源(platforms、rss-feeds、available-dates、keywords)-- **RSS 增强**:`get_latest_rss` 支持多日查询(days 参数),跨日期 URL 去重-- **正则匹配修复**:`get_trending_topics` 支持 `/pattern/` 正则语法和 `display_name`-- **缓存优化**:新增 `make_cache_key()` 函数,参数排序+MD5 哈希确保一致性-- **新增 check_version 工具**:支持同时检查 TrendRadar 和 MCP Server 版本更新---<details>-<summary>👉 点击展开:<strong>历史更新</strong></summary>+> 💡 详细配置教程见 [让 AI 帮我分析热点](#12-让-ai-帮我分析热点)### 2026/01/02 - v4.7.0@@ -322,7 +376,7 @@**🎉 重大更新:全面重构存储和核心架构**-- **多存储后端支持**:引入全新的存储模块,支持本地 SQLite 和远程云存储(S3 兼容协议,推荐免费的 Cloudflare R2),适应 GitHub Actions、Docker 和本地环境。+- **多存储后端支持**:引入全新的存储模块,支持本地 SQLite 和远程云存储(S3 兼容协议,例如 Cloudflare R2),适应 GitHub Actions、Docker 和本地环境。- **数据库结构优化**:重构 SQLite 数据库表结构,提升数据效率和查询能力。- **核心代码模块化**:将主程序逻辑拆分为 trendradar 包的多个模块,显著提升代码可维护性。- **增强功能**:实现日期格式标准化、数据保留策略、时区配置支持、时间显示优化,并修复远程存储数据持久化问题,确保数据合并的准确性。@@ -371,10 +425,10 @@**📖 文档优化**-- 新增 [报告配置](#7-报告配置) 章节:report 相关参数详解-- 新增 [推送时间窗口配置](#8-推送时间窗口配置) 章节:push_window 配置教程-- 新增 [执行频率配置](#9-执行频率配置) 章节:Cron 表达式说明和常用示例-- 新增 [多账号推送配置](#10-多账号推送配置) 章节:多账号推送配置详解+- 新增 [推送内容怎么显示?](#7-推送内容怎么显示) 章节:自定义推送样式和内容+- 新增 [什么时候给我推送?](#8-什么时候给我推送) 章节:设置推送时间段+- 新增 [多久运行一次?](#9-多久运行一次) 章节:设置自动运行频率+- 新增 [推送到多个群/设备](#10-推送到多个群设备) 章节:同时推送给多个接收者- 优化各配置章节:统一添加"配置位置"说明- 简化快速开始配置说明:三个核心文件一目了然- 优化 [Docker 部署](#6-docker-部署) 章节:新增镜像说明、推荐 git clone 部署、重组部署方式@@ -840,7 +894,7 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号| **内容顺序配置** | 调整"热点词汇统计"和"新增热点新闻"的显示顺序(v3.5.0 新增) | 统计在前 || **显示模式切换** | `keyword`=按关键词分组,`platform`=按平台分组(v4.6.0 新增) | keyword |-> 💡 详细配置教程见 [配置详解 - 报告配置](#7-报告配置) 和 [配置详解 - 推送时间窗口](#8-推送时间窗口配置)+> 💡 详细配置教程见 [推送内容怎么显示?](#7-推送内容怎么显示) 和 [什么时候给我推送?](#8-什么时候给我推送)### **精准内容筛选**@@ -862,7 +916,7 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号- **持续性分析**:区分一次性热点话题和持续发酵的深度新闻- **跨平台对比**:同一新闻在不同平台的排名表现,看出媒体关注度差异-> 💡 推送格式说明见 [配置详解 - 推送格式参考](#5-推送格式参考)+> 💡 推送格式说明见 [消息样式说明](#5-我收到的消息长什么样)### **个性化热点算法**@@ -874,7 +928,32 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号支持**企业微信**(+ 微信推送方案)、**飞书**、**钉钉**、**Telegram**、**邮件**、**ntfy**、**Bark**、**Slack**,消息直达手机和邮箱-> 💡 详细配置教程见 [配置详解 - 多账号推送配置](#10-多账号推送配置)+> 💡 详细配置教程见 [推送到多个群/设备](#10-推送到多个群设备)++### **AI 多语言翻译**(v5.2.0 新增)++将推送内容翻译为任意语言,打破语言壁垒,无论是阅读国内热点还是通过 RSS 订阅海外资讯,都能以母语轻松获取++- **一键翻译**:在 `config.yaml` 中设置 `ai_translation.enabled: true` 和目标语言即可+- **多语言支持**:支持 English、Korean、Japanese、French 等任意语言+- **智能批量处理**:自动批量翻译,减少 API 调用次数,节省成本+- **自定义风格**:通过 `ai_translation_prompt.txt` 自定义翻译风格和术语+- **共享模型配置**:与 AI 分析功能共用 `ai` 配置段的模型设置++```yaml+# config.yaml 快速启用示例+ai_translation:+ enabled: true+ language: "English" # 翻译目标语言+```++> 💡 翻译功能与 AI 分析功能共享模型配置,只需配置一次 `ai.api_key` 即可同时使用两个功能++**RSS 源参考**:以下是一些 RSS 订阅源合集,可按需选用+- [awesome-tech-rss](https://github.com/tuan3w/awesome-tech-rss) - 科技、创业、编程领域博客和媒体+- [awesome-rss-feeds](https://github.com/plenaryapp/awesome-rss-feeds) - 世界各国主流新闻媒体 RSS 合集++> ⚠️ 部分海外媒体内容可能涉及敏感话题,AI 模型可能拒绝翻译,建议根据实际需求筛选订阅源### **灵活存储架构**(v4.0.0 重大更新)@@ -883,7 +962,7 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号- **本地 SQLite 数据库**:Docker/本地环境默认,数据完全可控- **自动后端选择**:根据运行环境智能切换存储方式-> 💡 详细说明见 [配置详解 - 存储配置](#11-存储配置)+> 💡 详细说明见 [数据保存在哪里?](#11-数据保存在哪里)### **多端部署**- **GitHub Actions**:定时自动爬取 + 远程云存储(需签到续期)@@ -900,7 +979,7 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号- **灵活推送**:可选仅原始内容、仅 AI 分析、或两者都推送- **自定义提示词**:通过 `config/ai_analysis_prompt.txt` 自定义分析角度-> 💡 详细配置教程见 [AI 分析配置](#12-ai-分析配置)+> 💡 详细配置教程见 [让 AI 帮我分析热点](#12-让-ai-帮我分析热点)### **独立展示区(v5.0.0 新增)**@@ -910,7 +989,7 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号- **RSS 独立展示**:RSS 源内容可完整展示,不受关键词限制- **灵活配置**:支持配置展示平台、RSS 源、最大条数-> 💡 详细配置教程见 [报告配置 - 独立展示区](#7-报告配置)+> 💡 详细配置教程见 [推送内容怎么显示? - 独立展示区](#7-推送内容怎么显示)### **AI 智能分析(v3.0.0 新增)**@@ -922,15 +1001,15 @@ frequency_words.txt 文件增加了一个【必须词】功能,使用 + 号>> 详见 [AI 智能分析](#-ai-智能分析)-### **零技术门槛部署**+### **网页部署**-GitHub 一键 Fork 即可使用,无需编程基础。+运行后根目录生成 `index.html`,即为完整的新闻报告页面。-> 30秒部署: GitHub Pages(网页浏览)支持一键保存成图片,随时分享给他人+> **部署方式**:点击 **Use this template** 创建仓库,可部署到 Cloudflare Pages 或 GitHub Pages 等静态托管平台。>-> 1分钟部署: 企业微信(手机通知)--**💡 提示:** 想要**实时更新**的网页版?fork 后,进入你的仓库 Settings → Pages,启用 GitHub Pages。[效果预览](https://sansan0.github.io/TrendRadar/)。+> **💡 提示**:启用 GitHub Pages 可获得在线访问地址,进入仓库 Settings → Pages 即可开启。[效果预览](https://sansan0.github.io/TrendRadar/)+>+> ⚠️ 原 GitHub Actions 自动存储功能已下线(该方案曾导致 GitHub 服务器负载过高,影响平台稳定性)。### **减少 APP 依赖**@@ -941,9 +1020,9 @@ GitHub 一键 Fork 即可使用,无需编程基础。**典型场景:** 股市投资监控、品牌舆情追踪、行业动态关注、生活资讯获取-| Github Pages 效果(手机端适配、邮箱推送效果) | 飞书推送效果 |-|:---:|:---:|-|  |  |+| 网页效果(邮箱推送效果) | 飞书推送效果 | AI 分析推送效果 |+|:---:|:---:|:---:|+|  |  |  |<br>@@ -1050,9 +1129,11 @@ GitHub 一键 Fork 即可使用,无需编程基础。</details><details>- <summary>👉 点击展开:<strong>飞书机器人</strong>(消息显示最友好)</summary>+ <summary>👉 点击展开:<strong>飞书机器人</strong>(消息显示相对友好)</summary><br>+ 若启用 **AI 分析**,飞书推送偶发(约 5% 概率)会有数分钟延迟(推测为平台对 AI 生成内容的合规性审核)。+**GitHub Secret 配置(⚠️ Name 名称必须严格一致):**- **Name(名称)**:`FEISHU_WEBHOOK_URL`(请复制粘贴此名称,不要手打)- **Secret(值)**:你的飞书机器人 Webhook 地址(该链接开头类似 https://www.feishu.cn/flow/api/trigger-webhook/********)@@ -1103,7 +1184,7 @@ GitHub 一键 Fork 即可使用,无需编程基础。2. 点击"新建机器人应用"- 3. 进入创建的应用后,点击"流程涉及" > "创建流程" > "选择触发器"+ 3. 进入创建的应用后,点击"流程设计" > "创建流程" > "选择触发器"4. 往下滑动,点击"Webhook 触发"@@ -1741,7 +1822,7 @@ GitHub 一键 Fork 即可使用,无需编程基础。|-------------|-----------------|| `S3_REGION` | 区域(默认 `auto`,部分服务商可能需要指定) |- > 💡 **更多存储配置选项**:参见 [存储配置详解](#11-存储配置)+ > 💡 **更多存储配置选项**:参见 [数据保存在哪里?](#11-数据保存在哪里)<br>@@ -1788,7 +1869,7 @@ GitHub 一键 Fork 即可使用,无需编程基础。恭喜!现在你可以开始享受 TrendRadar 带来的高效信息流了。- 💬 **加入社区**:欢迎关注公众号「**[硅基茶水间](#问题答疑与交流)**」,分享你的使用心得和高级玩法。+ 💬 **加入社区**:欢迎关注公众号「**[硅基茶水间](#-支持项目)**」,分享你的使用心得和高级玩法。<br>@@ -1814,10 +1895,10 @@ GitHub 一键 Fork 即可使用,无需编程基础。> **📖 提醒**:本章节提供详细的配置说明,建议先完成 [快速开始](#-快速开始) 的基础配置,再根据需要回来查看详细选项。-### 1. 平台配置+### 1. 我要看哪些平台?<details id="自定义监控平台">-<summary>👉 点击展开:<strong>自定义监控平台</strong></summary>+<summary>👉 点击展开:<strong>选择资讯来源</strong></summary><br>**配置位置:** `config/config.yaml` 的 `platforms` 部分@@ -1843,9 +1924,9 @@ platforms:</details>-### 2. 关键词配置+### 2. 我关心什么内容?-在 `frequency_words.txt` 文件中配置监控的关键词,支持七种语法、区域标记和词组功能。+在 `frequency_words.txt` 文件中告诉机器人你想看什么,它就会帮你盯着。支持普通词、必须词、过滤词等多种玩法。| 语法类型 | 符号 | 作用 | 示例 | 匹配逻辑 ||---------|------|------|------|---------|@@ -2229,7 +2310,7 @@ report:</details>-### 3. 推送模式详解+### 3. 推送模式选哪个?<details><summary>👉 点击展开:<strong>三种推送模式详细对比</strong></summary>@@ -2242,8 +2323,6 @@ report:mode: "daily" # 可选: "daily" | "incremental" | "current"```-**Docker 环境变量:** `REPORT_MODE=incremental`-#### 详细对比表格| 模式 | 适用人群 | 推送时机 | 显示内容 | 典型使用场景 |@@ -2291,10 +2370,10 @@ report:</details>-### 4. 热点权重调整+### 4. 调整热点算法<details>-<summary>👉 点击展开:<strong>热点权重调整</strong></summary>+<summary>👉 点击展开:<strong>自定义热点权重</strong></summary><br>**配置位置:** `config/config.yaml` 的 `advanced.weight` 部分@@ -2340,10 +2419,10 @@ advanced:</details>-### 5. 推送格式参考+### 5. 我收到的消息长什么样?<details>-<summary>👉 点击展开:<strong>推送格式说明</strong></summary>+<summary>👉 点击展开:<strong>消息样式预览</strong></summary><br>#### 推送示例@@ -2474,16 +2553,12 @@ TrendRadar 提供两个独立的 Docker 镜像,可根据需求选择部署:| 环境变量 | 对应配置 | 示例值 | 说明 ||---------|---------|-------|------|- | `ENABLE_CRAWLER` | `advanced.crawler.enabled` | `true` / `false` | 是否启用爬虫 |- | `ENABLE_NOTIFICATION` | `notification.enabled` | `true` / `false` | 是否启用通知 |- | `REPORT_MODE` | `report.mode` | `daily` / `incremental` / `current`| 报告模式 |- | `DISPLAY_MODE` | `report.display_mode` | `keyword` / `platform` | 显示模式 || `ENABLE_WEBSERVER` | - | `true` / `false` | 是否自动启动 Web 服务器 || `WEBSERVER_PORT` | - | `8080` | Web 服务器端口 || `FEISHU_WEBHOOK_URL` | `notification.channels.feishu.webhook_url` | `https://...` | 飞书 Webhook(多账号用 `;` 分隔) || `AI_ANALYSIS_ENABLED` | `ai_analysis.enabled` | `true` / `false` | 是否启用 AI 分析(v5.0.0 新增) |- | `AI_API_KEY` | `ai_analysis.api_key` | `sk-xxx...` | AI API Key(v5.0.0 新增) |- | `AI_PROVIDER` | `ai_analysis.provider` | `deepseek` / `openai` / `gemini` | AI 提供商(v5.0.0 新增) |+ | `AI_API_KEY` | `ai.api_key` | `sk-xxx...` | AI API Key(ai_analysis 和 ai_translation 共享) |+ | `AI_PROVIDER` | `ai.provider` | `deepseek` / `openai` / `gemini` | AI 提供商 || `S3_*` | `storage.remote.*` | - | 远程存储配置(5 个参数) |**配置优先级**:环境变量 > config.yaml@@ -2796,10 +2871,10 @@ MCP 服务启动后,根据不同客户端进行配置:</details>-### 7. 报告配置+### 7. 推送内容怎么显示?<details>-<summary>👉 点击展开:<strong>报告相关参数配置</strong></summary>+<summary>👉 点击展开:<strong>自定义推送样式和内容</strong></summary><br>**配置位置:** `config/config.yaml` 的 `report` 部分@@ -2814,119 +2889,64 @@ report:reverse_content_order: false # 内容顺序配置```-#### 配置项详解+#### 常用配置项说明-| 配置项 | 类型 | 默认值 | 说明 |-|-------|------|-------|------|-| `mode` | string | `daily` | 推送模式,可选 `daily`/`incremental`/`current`,详见 [推送模式详解](#3-推送模式详解) |-| `display_mode` | string | `keyword` | 显示模式,可选 `keyword`/`platform`,详见下方说明 |-| `rank_threshold` | int | `5` | 排名高亮阈值,排名 ≤ 该值的新闻会加粗显示 |-| `sort_by_position_first` | bool | `false` | 排序优先级:`false`=按热点条数排序,`true`=按配置位置排序 |-| `max_news_per_keyword` | int | `0` | 每个关键词最大显示数量,`0`=不限制 |-| `reverse_content_order` | bool | `false` | 内容顺序:`false`=热点词汇统计在前,`true`=新增热点新闻在前 |+| 我想调整什么 | 修改哪个参数 | 默认值 | 说明 |+|-------------|-------------|-------|------|+| **推送模式** | `mode` | `daily` | 决定推送时机和内容,详见 [推送模式详解](#3-推送模式详解) |+| **分组方式** | `display_mode` | `keyword` | `keyword`=按关键词分组(如"AI"),`platform`=按平台分组(如"微博") |+| **高亮重点** | `rank_threshold` | `5` | 排名在前 5 的新闻会**加粗**显示,一眼看到最火的 |+| **排序规则** | `sort_by_position_first` | `false` | `false`=热度高的排前面,`true`=你配置的词排前面 |+| **数量限制** | `max_news_per_keyword` | `0` | 每个关键词最多看几条?`0`表示不限制 |+| **显示顺序** | `reverse_content_order` | `false` | `false`=先看统计,`true`=先看新增新闻 |-#### 显示模式配置(v4.6.0 新增)+#### 分组方式对比(display_mode)-控制推送消息和 HTML 报告中新闻的分组方式:+你是想看"这个话题下有哪些新闻",还是"这个平台上有哪些新闻"?| 模式 | 分组方式 | 标题前缀 | 适用场景 ||------|---------|---------|---------|-| `keyword`(默认) | 按关键词分组 | `[平台名]` | 关注特定话题的用户 |-| `platform` | 按平台分组 | `[关键词]` | 关注特定平台的用户 |--**示例对比:**--```-# keyword 模式(按关键词分组)-📊 热点词汇统计-🔥 [1/3] AI : 12 条- 1. [微博] OpenAI发布GPT-5 #1-#3 - 08:30 (5次)- 2. [知乎] 如何看待AI取代程序员 #2 - 09:15 (3次)--# platform 模式(按平台分组)-📊 热点新闻统计-🔥 [1/4] 微博 : 12 条- 1. [AI] OpenAI发布GPT-5 #1-#3 - 08:30 (5次)- 2. [特朗普] 特朗普宣布重大政策 #2 - 09:15 (3次)-```--**Docker 环境变量:**-```bash-DISPLAY_MODE=platform-```--#### 内容顺序配置(v3.5.0 新增)--控制推送消息和 HTML 报告中两部分内容的显示顺序:--| 配置值 | 显示顺序 |-|-------|---------|-| `false`(默认) | ① 热点词汇统计 → ② 新增热点新闻 |-| `true` | ① 新增热点新闻 → ② 热点词汇统计 |--**适用场景:**-- `false`(默认):适合关注关键词匹配结果的用户,先看分类统计-- `true`:适合关注最新动态的用户,优先查看新增热点--**Docker 环境变量:**-```bash-REVERSE_CONTENT_ORDER=true-```+| `keyword`(默认) | **按关键词聚合** | `[平台名]` | 我关注"AI",想看各平台关于AI的新闻 |+| `platform` | **按平台聚合** | `[关键词]` | 我关注"微博",想看微博上关于我关注词的新闻 |-#### 排序优先级配置--**示例场景:** 配置顺序 A、B、C,热点数 A(3条)、B(10条)、C(5条)+#### 内容显示顺序(reverse_content_order)| 配置值 | 显示顺序 | 适用场景 ||-------|---------|---------|-| `false`(默认) | B(10条) → C(5条) → A(3条) | 关注热度趋势 |-| `true` | A(3条) → B(10条) → C(5条) | 关注个人优先级 |--**Docker 环境变量:**-```bash-SORT_BY_POSITION_FIRST=true-MAX_NEWS_PER_KEYWORD=10-```--#### 独立展示区配置(v5.0.0 新增)+| `false`(默认) | ① 📊 热点统计 (聚合) → ② 🆕 新增新闻 (流水) | **推荐**:先看整体热度,再看具体新增 |+| `true` | ① 🆕 新增新闻 (流水) → ② 📊 热点统计 (聚合) | **追新党**:只想第一时间看到新出了什么 |-为指定平台提供完整热榜展示,不受 `frequency_words.txt` 关键词过滤影响。+#### 排序优先级(sort_by_position_first)-**配置位置:** `config/config.yaml` 的 `notification.standalone_display` 部分+假设你配置了关键词:1.特斯拉,2.比亚迪。+实际热度:比亚迪(10条),特斯拉(3条)。-```yaml-notification:- standalone_display:- enabled: false # 是否启用- platforms: ["zhihu", "weibo"] # 热榜平台 ID 列表- rss_feeds: ["hacker-news"] # RSS 源 ID 列表- max_items: 20 # 每个源最多展示条数(0=不限制)-```+| 配置值 | 排序结果 | 你的想法 |+|-------|---------|---------|+| `false`(默认) | 比亚迪(10条) → 特斯拉(3条) | "谁火谁排前面" |+| `true` | 特斯拉(3条) → 比亚迪(10条) | "我配置的顺序就是优先级,不管它火不火" |-**适用场景:**-- 想完整查看某个平台(如知乎)的热榜排名,而不是只看匹配关键词的内容-- 订阅了更新较少的 RSS 源(如个人博客),希望每次更新都完整推送+#### 独立展示区(standalone)-**效果示例:**-```-📋 独立展示区 (共 15 条)+**场景**:有些平台(比如知乎热榜、HackerNews),我想**完整看一遍**,不管有没有匹配我的关键词。-知乎热榜 (10 条):- 1. [知乎] 如何看待 OpenAI 发布 Sora?- 2. [知乎] 2024 年考研分数线公布...- ...+```yaml+display:+ regions:+ standalone: true # 开启这个“特权区域”-Hacker News (5 条):- 1. [Hacker News] Launch HN: TrendRadar...- ...+ standalone:+ platforms: ["zhihu", "weibo"] # 这些平台的热榜给我完整显示+ rss_feeds: ["hacker-news"] # 这些RSS源的内容给我完整显示+ max_items: 20 # 最多显示多少条```</details>-### 8. 推送时间窗口配置+### 8. 什么时候给我推送?<details>-<summary>👉 点击展开:<strong>推送时间窗口控制详解</strong></summary>+<summary>👉 点击展开:<strong>设置推送时间段</strong></summary><br>**配置位置:** `config/config.yaml` 的 `notification.push_window` 部分@@ -2999,10 +3019,10 @@ notification:</details>-### 9. 执行频率配置+### 9. 多久运行一次?<details>-<summary>👉 点击展开:<strong>自动运行频率设置</strong></summary>+<summary>👉 点击展开:<strong>设置自动运行频率</strong></summary><br>**配置位置:** `.github/workflows/crawler.yml` 的 `schedule` 部分@@ -3013,526 +3033,208 @@ on:- cron: "0 * * * *" # 每小时运行一次```-#### 什么是 Cron 表达式?+#### 怎么修改运行频率?-Cron 是一种定时任务格式,由 5 个部分组成:`分 时 日 月 周`+GitHub Actions 使用一种叫 "Cron" 的时间格式,不需要深入理解,直接复制下面的代码替换即可。-```-┌───────────── 分钟 (0-59)-│ ┌───────────── 小时 (0-23)-│ │ ┌───────────── 日期 (1-31)-│ │ │ ┌───────────── 月份 (1-12)-│ │ │ │ ┌───────────── 星期 (0-6,0=周日)-│ │ │ │ │-* * * * *-```--#### 常用配置示例+**配置位置:** `.github/workflows/crawler.yml` 文件中的 `schedule` 部分-| 想要的效果 | Cron 表达式 | 说明 |+| 我想要... | 复制这行代码 | 说明 ||-----------|------------|------|-| 每小时运行 | `0 * * * *` | 每小时的第 0 分钟运行(默认) |-| 每 30 分钟运行 | `*/30 * * * *` | 每隔 30 分钟运行一次 |-| 每天早 8 点运行 | `0 0 * * *` | UTC 0:00 = 北京时间 8:00 |-| 工作时间运行 | `*/30 0-14 * * *` | 北京 8:00-22:00,每 30 分钟 |-| 每天 3 次 | `0 0,6,12 * * *` | 北京 8:00、14:00、20:00 |+| **每小时一次** | `- cron: "0 * * * *"` | **默认配置**,第 0 分钟运行 |+| **每 30 分钟** | `- cron: "*/30 * * * *"` | 每隔 30 分钟运行一次 |+| **每天早 8 点** | `- cron: "0 0 * * *"` | ⚠️ 写 `0` 是因为 UTC 时间 (0点) = 北京时间 (8点) |+| **工作时间每半小时** | `- cron: "*/30 0-14 * * *"` | 对应北京时间 8:00 - 22:00 |+| **一日三餐点** | `- cron: "0 0,6,12 * * *"` | 对应北京时间 8:00、14:00、20:00 |-#### 重要提示+#### ⚠️ 两个重要提醒-> ⚠️ **时区注意**:GitHub Actions 使用 **UTC 时间**,北京时间需要 **减 8 小时**-> - 想要北京时间 8:00 运行 → 设置 UTC 0:00-> - 想要北京时间 20:00 运行 → 设置 UTC 12:00+1. **时差问题**:GitHub 的服务器在国外,用的是 UTC 时间。+ - **简单的算术题**:你想设定的北京时间 **减去 8 小时** = 你要填的时间。+ - *例子:想让它北京时间 20:00 运行,设置里要填 12:00*-> ⚠️ **频率限制**:GitHub 对每个账号的 Actions 运行次数有限额-> - **建议**:不要设置比 30 分钟更短的间隔-> - **原因**:过于频繁可能被判定为滥用,面临封号风险-> - **实际情况**:GitHub Actions 执行时间本身就有偏差,设置太精确意义不大+2. **不要太频繁**:建议间隔不要少于 30 分钟。+ - GitHub 免费资源有限,跑得太勤可能会被官方限制账号。+ - 而且 Actions 启动本身就有几分钟延迟,太精确的控制没有意义。-#### 修改方法+#### 手把手修改步骤-1. 打开你 fork 的仓库-2. 找到 `.github/workflows/crawler.yml` 文件-3. 点击编辑(铅笔图标)-4. 修改 `cron: "0 * * * *"` 中的表达式-5. 点击 "Commit changes" 保存+1. 在你的 GitHub 仓库中,找到 `.github/workflows/crawler.yml` 文件+2. 点击右上角的 ✏️ (Edit) 按钮+3. 找到 `cron: "..."` 那一行,把引号里的内容换成上面的"代码"+4. 点击右上角的绿色 **Commit changes** 按钮保存</details>-### 10. 多账号推送配置+### 10. 推送到多个群/设备<details>-<summary>👉 点击展开:<strong>多账号推送配置详解</strong></summary>+<summary>👉 点击展开:<strong>同时推送给多个接收者</strong></summary>-> ### ⚠️ **安全警告**-> **GitHub Fork 用户请勿在 `config.yaml` 中配置推送信息!**+> ### ⚠️ **安全第一**+> **不要在 `config.yaml` 里直接写密码/Token!**+> 如果你把包含密码的文件上传到 GitHub,全世界都能看到。>-> - **风险说明**:`config.yaml` 会被提交到公开的 Git 仓库,配置推送信息(Webhook URL、Token 等)会泄露敏感数据-> - **推荐方式**:-> - **GitHub Actions 用户** → 使用 GitHub Secrets 环境变量-> - **Docker 用户** → 使用 [`.env` 文件配置](#6-docker-部署)(`.env` 已在 `.gitignore` 中,不会被提交)-> - **本地开发用户**:可以在 `config.yaml` 中配置(确保不会 push 到公开仓库)--#### 多账号推送说明(v3.5.0 新增)--- **支持多账号配置**:所有推送渠道(飞书、钉钉、企业微信、Telegram、ntfy、Bark、Slack)均支持配置多个账号-- **配置方式**:使用英文分号 `;` 分隔多个账号值-- **示例**:`FEISHU_WEBHOOK_URL` 的 Secret 值填写 `https://webhook1;https://webhook2`-- **配对配置**:Telegram 和 ntfy 需要保证配对参数数量一致(如 token 和 chat_id 都是 2 个)-- **数量限制**:默认每个渠道最多 3 个账号,超出部分被截断--**多账号配置示例**:--| Name(名称) | Secret(值)示例 |-|-------------|-----------------|-| `FEISHU_WEBHOOK_URL` | `https://webhook1;https://webhook2;https://webhook3` |-| `TELEGRAM_BOT_TOKEN` | `token1;token2` |-| `TELEGRAM_CHAT_ID` | `chatid1;chatid2` |-| `NTFY_TOPIC` | `topic1;topic2` |-| `NTFY_TOKEN` | `;token2`(第一个无 token 时留空占位) |--#### 支持的渠道--| 渠道 | 配置项 | 是否需要配对 | 说明 |-|------|--------|-------------|------|-| **飞书** | `feishu_url` | 否 | 多个 webhook URL |-| **钉钉** | `dingtalk_url` | 否 | 多个 webhook URL |-| **企业微信** | `wework_url` | 否 | 多个 webhook URL |-| **Telegram** | `telegram_bot_token` + `telegram_chat_id` | ✅ 是 | token 和 chat_id 数量必须一致 |-| **ntfy** | `ntfy_topic` + `ntfy_token` | ✅ 是 | topic 和 token 数量必须一致(token 可选) |-| **Bark** | `bark_url` | 否 | 多个推送 URL |-| **Slack** | `slack_webhook_url` | 否 | 多个 webhook URL |-| **邮件** | `email_to` | - | 已支持多收件人(逗号分隔),无需修改 |--#### 推荐配置方式 1:GitHub Actions 环境变量--**配置位置**:GitHub Repo → Settings → Secrets and variables → Actions → Repository secrets--**基础配置示例**:-```bash-# 多账号数量限制-MAX_ACCOUNTS_PER_CHANNEL=3+> **正确做法**:+> - **GitHub Actions 用户**:去 Settings -> Secrets 里添加+> - **Docker 用户**:写在 `.env` 文件里(这个文件不会被上传)-# 飞书多账号(3个群组)-FEISHU_WEBHOOK_URL=https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy;https://hook3.feishu.cn/zzz+#### 怎么同时推送到多个地方?-# 钉钉多账号(2个群组)-DINGTALK_WEBHOOK_URL=https://oapi.dingtalk.com/xxx;https://oapi.dingtalk.com/yyy+很简单,在配置时用分号 `;` 把多个地址隔开就行了。-# 企业微信多账号(2个群组)-WEWORK_WEBHOOK_URL=https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx;https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yyy+**举个例子**:+假设你有两个飞书群,想同时收到推送:+- 群1地址:`https://.../webhook/aaa`+- 群2地址:`https://.../webhook/bbb`-# Bark多账号(2个设备)-BARK_URL=https://api.day.app/key1;https://api.day.app/key2+配置时填写:+`https://.../webhook/aaa;https://.../webhook/bbb`-# Slack多账号(2个频道)-SLACK_WEBHOOK_URL=https://hooks.slack.com/xxx;https://hooks.slack.com/yyy-```--**配对配置示例(Telegram 和 ntfy)**:+#### 支持多账号的平台-<details>-<summary><strong>Telegram 配对配置</strong></summary>+| 平台 | 配置方法 | 注意事项 |+|------|---------|----------|+| **飞书/钉钉/企微** | 用 `;` 分隔多个 Webhook URL | 最简单,直接串起来就行 |+| **Bark (iOS)** | 用 `;` 分隔多个 Key URL | 推送到多台 iPhone |+| **Telegram** | Token 和 ChatID 都要用 `;` 分隔 | ⚠️ **注意顺序要对应**:<br>Token1 对应 ChatID1<br>Token2 对应 ChatID2 |+| **ntfy** | Topic 和 Token 都要用 `;` 分隔 | 如果某个Topic不需要Token,留空即可:<br>`token1;;token3` (中间那个是空的) |-```bash-# ✅ 正确配置:2个token对应2个chat_id-TELEGRAM_BOT_TOKEN=123456:AAA-BBB;789012:CCC-DDD-TELEGRAM_CHAT_ID=-100111;-100222--# ❌ 错误配置:数量不一致,将跳过推送-TELEGRAM_BOT_TOKEN=token1;token2;token3-TELEGRAM_CHAT_ID=id1;id2-```--**说明**:`token` 和 `chat_id` 的数量必须完全一致,否则该渠道推送会被跳过。--</details>--<details>-<summary><strong>ntfy 配对配置</strong></summary>+#### 常用配置示例 (GitHub Secrets / .env)```bash-# ✅ 正确配置:3个topic,只有第2个需要token-NTFY_TOPIC=topic1;topic2;topic3-NTFY_TOKEN=;token_for_topic2;+# 飞书发给 3 个群+FEISHU_WEBHOOK_URL=https://hook1...;https://hook2...;https://hook3...-# ✅ 正确配置:2个topic都需要token-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2+# 钉钉发给 2 个群+DINGTALK_WEBHOOK_URL=https://oapi...;https://oapi...-# ❌ 错误配置:topic和token数量不匹配-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2;token3+# Telegram 发给 2 个人 (注意一一对应)+TELEGRAM_BOT_TOKEN=tokenA;tokenB+TELEGRAM_CHAT_ID=userA;userB```-**说明**:-- 如果某个 topic 不需要 token,在对应位置留空(两个分号之间)-- `topic` 和 `token` 的数量必须一致--</details>-------#### 推荐配置方式 2:Docker 环境变量(.env)--**配置位置**:项目根目录 `docker/.env` 文件--**基础配置示例**:-```bash-# 多账号数量限制-MAX_ACCOUNTS_PER_CHANNEL=3--# 飞书多账号(3个群组)-FEISHU_WEBHOOK_URL=https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy;https://hook3.feishu.cn/zzz--# 钉钉多账号(2个群组)-DINGTALK_WEBHOOK_URL=https://oapi.dingtalk.com/xxx;https://oapi.dingtalk.com/yyy--# 企业微信多账号(2个群组)-WEWORK_WEBHOOK_URL=https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxx;https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yyy--# Bark多账号(2个设备)-BARK_URL=https://api.day.app/key1;https://api.day.app/key2--# Slack多账号(2个频道)-SLACK_WEBHOOK_URL=https://hooks.slack.com/xxx;https://hooks.slack.com/yyy-```--**配对配置示例(Telegram 和 ntfy)**:--<details>-<summary><strong>Telegram 配对配置</strong></summary>--```bash-# ✅ 正确配置:2个token对应2个chat_id-TELEGRAM_BOT_TOKEN=123456:AAA-BBB;789012:CCC-DDD-TELEGRAM_CHAT_ID=-100111;-100222--# ❌ 错误配置:数量不一致,将跳过推送-TELEGRAM_BOT_TOKEN=token1;token2;token3-TELEGRAM_CHAT_ID=id1;id2-```--**说明**:`token` 和 `chat_id` 的数量必须完全一致,否则该渠道推送会被跳过。+> **提示**:为了防止滥用,默认限制每个平台最多推送到 3 个账号。如果需要更多,可以修改 `MAX_ACCOUNTS_PER_CHANNEL` 配置。</details>-<details>-<summary><strong>ntfy 配对配置</strong></summary>--```bash-# ✅ 正确配置:3个topic,只有第2个需要token-NTFY_TOPIC=topic1;topic2;topic3-NTFY_TOKEN=;token_for_topic2;--# ✅ 正确配置:2个topic都需要token-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2--# ❌ 错误配置:topic和token数量不匹配-NTFY_TOPIC=topic1;topic2-NTFY_TOKEN=token1;token2;token3-```--**说明**:-- 如果某个 topic 不需要 token,在对应位置留空(两个分号之间)-- `topic` 和 `token` 的数量必须一致--</details>-------#### 推送行为说明--1. **独立推送**:每个账号独立发送,一个失败不影响其他账号-2. **部分成功判定**:只要有一个账号发送成功,整体视为成功-3. **日志区分**:多账号时日志会显示"账号1"、"账号2"等标签-4. **批次间隔**:多账号会增加总发送时间(每个账号独立计算批次间隔)-------#### 常见问题--<details>-<summary><strong>Q1: 超过 3 个账号会怎样?</strong></summary>-<br>--系统会自动截断到配置的最大数量,并输出警告日志。可通过 `max_accounts_per_channel` 调整限制。--**⚠️ GitHub Actions 用户特别注意**:-- **不建议配置过多账号**(建议不超过 3 个),可能导致:- - **触发 GitHub Actions 速率限制**:频繁的网络请求可能被识别为异常行为- - **潜在账号风险**:过度使用 GitHub Actions 资源可能影响账号状态--</details>--<details>-<summary><strong>Q2: 多账号会影响推送速度吗?</strong></summary>-<br>--会。每个账号独立发送,总时间 = 账号数 × 单账号发送时间。建议控制账号数量。--</details>--<details>-<summary><strong>Q3: 本地开发用户如何在 config.yaml 中配置?</strong></summary>-<br>--如果你是本地开发且**不会将代码推送到公开仓库**,可以直接在 `config/config.yaml` 中配置:--```yaml-notification:- enabled: true-- channels:- feishu:- webhook_url: "https://hook1.feishu.cn/xxx;https://hook2.feishu.cn/yyy"- telegram:- bot_token: "token1;token2"- chat_id: "id1;id2"--advanced:- max_accounts_per_channel: 3-```--**⚠️ 重要提醒**:-- 确保 `config/config.yaml` 在 `.gitignore` 中(如果会提交代码)-- 或者只在本地开发环境使用,**绝不提交到公开仓库**--</details>--</details>--### 11. 存储配置+### 11. 数据保存在哪里?<details id="storage-config">-<summary>👉 点击展开:<strong>存储架构配置详解</strong></summary>+<summary>👉 点击展开:<strong>选择数据存储位置</strong></summary><br>-#### 存储后端选择--**配置位置**:`config/config.yaml` 的 `storage` 部分+#### 数据会存在哪里?-v4.0.0 版本重构了存储架构,支持多种存储后端:+系统会自动帮你选择最合适的地方,你通常不需要操心:-```yaml-storage:- backend: auto # 存储后端:auto(自动选择)/ local(本地SQLite)/ remote(远程云存储)+| 你的运行环境 | 数据存在哪 | 说明 |+|-------------|-----------|------|+| **Docker / 本地运行** | **本地硬盘** | 存在项目目录下的 `output/` 文件夹里,随时可以查看。 |+| **GitHub Actions** | **云端存储** | 因为 GitHub Actions 运行完就会销毁环境,所以必须配置云存储(例如 Cloudflare R2)。 |- formats:- sqlite: true # 是否启用SQLite存储- txt: true # 是否生成TXT快照- html: true # 是否生成HTML报告+#### 怎么配置云存储?(GitHub Actions 用户必看)- local:- data_dir: "output" # 本地存储目录- retention_days: 0 # 本地数据保留天数,0表示永久保留+如果你是用 GitHub Actions 运行,你需要一个"云端硬盘"来存数据。例如使用 Cloudflare R2(因为有免费额度)。- remote:- endpoint_url: "" # S3 API 端点- bucket_name: "" # 存储桶名称- access_key_id: "" # 访问密钥ID- secret_access_key: "" # 访问密钥- region: "" # 区域(可选)- retention_days: 0 # 远程数据保留天数,0表示永久保留-- pull:- enabled: false # 是否启用启动时从远程拉取数据- days: 7 # 拉取最近N天的数据-```+**在 GitHub Secrets 里添加这 5 个变量:**-#### 后端选择策略+| 变量名 | 填什么 |+|-------|-------|+| `STORAGE_BACKEND` | `remote` |+| `S3_BUCKET_NAME` | 你的存储桶名字 |+| `S3_ACCESS_KEY_ID` | 你的 Access Key |+| `S3_SECRET_ACCESS_KEY` | 你的 Secret Key |+| `S3_ENDPOINT_URL` | 你的 R2 接口地址 |-| backend 值 | 说明 | 适用场景 |-|-----------|------|---------|-| `auto` | **自动选择**(推荐) | 根据运行环境智能选择:<br>• GitHub Actions → Remote<br>• Docker/本地 → Local |-| `local` | 本地 SQLite 数据库 | Docker 部署、本地开发 |-| `remote` | 远程云存储(S3 兼容,如 Cloudflare R2) | GitHub Actions、多机器同步 |+> 💡 **详细教程**:怎么申请 R2?请看 [快速开始 - 远程存储配置](#-快速开始)+#### 数据会保存多久?-#### 远程云存储配置+默认情况下,我们不会自动删除你的数据。但如果你觉得数据太多占空间,可以设置"自动清理"。-**环境变量**(推荐方式):--```bash-# GitHub Actions / Docker 环境变量-STORAGE_BACKEND=remote # 或 auto--# 本地/远程数据保留天数(0 表示永久保留)-LOCAL_RETENTION_DAYS=0-REMOTE_RETENTION_DAYS=0--# S3 兼容存储配置(以 Cloudflare R2 为例)-S3_BUCKET_NAME=your-bucket-name-S3_ACCESS_KEY_ID=your-access-key-id-S3_SECRET_ACCESS_KEY=your-secret-access-key-S3_ENDPOINT_URL=https://<account-id>.r2.cloudflarestorage.com-S3_REGION=auto--# 数据拉取配置(可选,从远程同步到本地)-PULL_ENABLED=false-PULL_DAYS=7-```--**获取凭据**:参见 [快速开始 - 远程存储配置](#-快速开始)--#### 数据清理策略--**自动清理**:每次运行结束时检查并删除超过保留天数的数据。+**配置位置**:`config/config.yaml````yamlstorage:local:- retention_days: 30 # 本地保留最近30天数据+ retention_days: 30 # 本地数据只保留 30 天 (0 表示永久)remote:- retention_days: 30 # 远程保留最近30天数据+ retention_days: 30 # 云端数据只保留 30 天```-**清理逻辑**:-- 本地存储:删除过期日期的数据库文件(如 `output/news/2025-11-10.db`、`output/rss/2025-11-10.db`)-- 远程存储:批量删除过期的云端对象(如 `news/2025-11-10.db`、`rss/2025-11-10.db`)+#### 推送时间不对?(时区设置)-#### 时区配置(v4.0.0 新增)+如果你身在海外,或者发现推送时间跟你的本地时间对不上,可以修改时区。-**全球时区支持**:解决非中国用户推送时间窗口问题。+**配置位置**:`config/config.yaml````yamlapp:- timezone: "Asia/Shanghai" # 默认中国时区- # 其他示例:- # timezone: "America/Los_Angeles" # 美西时间- # timezone: "Europe/London" # 英国时间+ timezone: "Asia/Shanghai" # 默认是中国时间```--**支持所有 IANA 时区名称**:[时区列表](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)---#### 不兼容变更--⚠️ **v4.0.0 不兼容 v3.x 数据**:--1. 数据库结构完全重构,无法读取旧数据-2. 文件路径格式变更(ISO 格式)--**迁移建议**:-- 从 v4.0.0 开始重新收集数据-- 旧数据如需保留,请手动重命名目录格式(不推荐)+- 比如你在美国洛杉矶,改成:`America/Los_Angeles`+- 比如你在英国伦敦,改成:`Europe/London`</details>-### 12. AI 分析配置+### 12. 让 AI 帮我分析热点<details id="ai-analysis-config">-<summary>👉 点击展开:<strong>AI 分析推送配置详解</strong></summary>+<summary>👉 点击展开:<strong>开启 AI 智能分析功能</strong></summary><br>-#### 功能概述--v5.0.0 新增 AI 分析推送功能,使用 AI 大模型对推送内容进行深度分析,自动生成热点洞察报告。--**分析内容包括**:-- 热点趋势概述-- 关键词热度分析-- 跨平台关联分析-- 潜在影响评估-- 值得关注的信号-- 总结与建议--#### 配置位置--**配置文件**:`config/config.yaml` 的 `ai_analysis` 部分+#### AI 能帮我做什么?-```yaml-ai_analysis:- enabled: false # 是否启用 AI 分析- provider: "deepseek" # AI 提供商- api_key: "" # API Key(建议使用环境变量 AI_API_KEY)- model: "deepseek-chat" # 模型名称- base_url: "" # 自定义 API 端点(可选)- timeout: 90 # 请求超时(秒)- push_mode: "both" # 推送模式- max_news_for_analysis: 50 # 最多分析多少条新闻- include_rss: true # 是否包含 RSS 内容- prompt_file: "ai_analysis_prompt.txt" # 提示词配置文件-```--#### 支持的 AI 提供商--| 提供商 | provider 值 | 默认端点 |-|-------|------------|---------|-| **DeepSeek** | `deepseek` | https://api.deepseek.com/v1/chat/completions |-| **OpenAI** | `openai` | https://api.openai.com/v1/chat/completions |-| **Google Gemini** | `gemini` | https://generativelanguage.googleapis.com/v1beta/openai/chat/completions |-| **自定义** | `custom` | 需填写 base_url |+开启这个功能后,AI 会像一个专业的分析师,在推送每一批新闻时:+1. **自动阅读**:阅读所有匹配到的热点新闻+2. **深度思考**:分析原本孤立的新闻之间的关联+3. **撰写报告**:在推送消息的末尾,附上一份简短深刻的"洞察报告"-> 💡 **提示**:使用 `custom` 提供商时,`base_url` 需填写完整的 API 地址(如 `https://api.example.com/v1/chat/completions`)+**包含内容**:热点趋势总结、舆论风向判断、跨平台关联分析、潜在影响评估等。-#### 推送模式说明+#### 怎么开启 AI 分析?-| 模式 | 说明 |-|------|------|-| `only_analysis` | 仅推送 AI 分析结果,不推送原始内容 |-| `both` | 两者都推送(默认),AI 分析追加在原始内容后 |+最简单的方法是通过环境变量配置(推荐 GitHub Secrets 或 .env)。-> 💡 **提示**:如果不需要 AI 分析功能,请将 `enabled` 设为 `false`,无需使用 `push_mode` 控制+**必需的配置项**:-#### 环境变量支持+| 变量名 | 填什么 | 说明 |+|-------|-------|------|+| `AI_ANALYSIS_ENABLED` | `true` | 开启开关 |+| `AI_API_KEY` | `sk-xxxxxx` | 你的 API Key |+| `AI_PROVIDER` | `deepseek` | AI 提供商(见下表) |+| `AI_MODEL` | `deepseek-chat` | 模型名称 |-| 环境变量 | 说明 | 示例 |-|---------|------|------|-| `AI_ANALYSIS_ENABLED` | 是否启用 AI 分析 | `true` / `false` |-| `AI_API_KEY` | AI API Key | `sk-xxx...` |-| `AI_PROVIDER` | AI 提供商 | `deepseek` / `openai` / `gemini` / `custom` |-| `AI_MODEL` | 模型名称 | `deepseek-chat` |-| `AI_BASE_URL` | 完整 API 地址(custom 提供商必填) | `https://api.example.com/v1/chat/completions` |--#### 自定义提示词+**支持的 AI 提供商**:-编辑 `config/ai_analysis_prompt.txt` 文件可自定义 AI 分析角色和输出格式。--**文件结构**:-```-[system]-系统提示词,定义 AI 角色和分析原则-...+| 提供商 | AI_PROVIDER 填什么 | 默认模型 (AI_MODEL) |+|-------|-------------------|-------------------|+| **DeepSeek** (推荐) | `deepseek` | `deepseek-chat` |+| **OpenAI** | `openai` | `gpt-4o` |+| **Google Gemini** | `gemini` | `gemini-1.5-flash` |+| **自定义** (OneAPI) | `custom` | 需额外配置 `AI_BASE_URL` |-[user]-用户提示词模板,支持变量替换-...-```+> 💡 **小技巧**:DeepSeek 性价比极高,非常适合用来做这种高频的新闻分析。-**可用变量**:-- `{report_mode}` - 当前报告模式-- `{report_type}` - 报告类型描述-- `{current_time}` - 当前时间-- `{news_count}` - 热榜新闻条数-- `{rss_count}` - RSS 新闻条数-- `{keywords}` - 匹配的关键词列表-- `{platforms}` - 数据来源平台列表-- `{news_content}` - 新闻内容+#### 进阶玩法:AI 翻译-#### 快速启用示例+如果你关注了国外的 RSS 源(比如 Hacker News),AI 可以帮你把内容翻译成中文推送。-**方式一:配置文件**+**配置位置**:`config/config.yaml````yaml-ai_analysis:- enabled: true- provider: "deepseek"- api_key: "sk-your-api-key"- model: "deepseek-chat"- push_mode: "both"+ai_translation:+ enabled: true # 开启翻译+ language: "Chinese" # 翻译成什么语言 (Chinese, English, Japanese...)```-**方式二:环境变量(推荐)**+#### 进阶玩法:自定义 AI "人设"-```bash-# GitHub Actions: 添加到 Secrets-# Docker: 添加到 .env 文件-AI_ANALYSIS_ENABLED=true-AI_API_KEY=sk-your-api-key-AI_PROVIDER=deepseek-AI_MODEL=deepseek-chat-```+觉得 AI 说话太官方?你可以修改它的提示词,让它变成你喜欢的风格(比如"毒舌评论员"、"资深投资顾问")。++- **修改文件**:`config/ai_analysis_prompt.txt`+- **修改方法**:直接用记事本打开编辑,告诉 AI 你想要什么样的分析风格。</details>@@ -3569,7 +3271,7 @@ AI 分析功能**不是**直接查询网络实时数据,而是分析你**本Cherry Studio 提供 GUI 配置界面,5 分钟快速部署,复杂的部分是一键安装的。-**图文部署教程**:现已更新到我的[公众号](#问题答疑与交流),回复 "mcp" 即可+**图文部署教程**:现已更新到我的[公众号](#-支持项目),回复 "mcp" 即可**详细部署教程**:[README-Cherry-Studio.md](README-Cherry-Studio.md)@@ -3583,7 +3285,7 @@ Cherry Studio 提供 GUI 配置界面,5 分钟快速部署,复杂的部分> 💡 **提示**:实际不建议一次性问多个问题。如果你选择的 AI 模型连下图的按顺序调用都无法做到,建议换一个。-<img src="/_image/ai2.png" alt="mcp 使用效果图" width="600">+<img src="/_image/ai4.png" alt="mcp 使用效果图" width="600"><br>@@ -3893,27 +3595,6 @@ MCP Inspector 是官方调试工具,用于测试 MCP 连接:</details>-<br>--## ☕问题答疑与交流--> 如果本项目对你有帮助,你可以选择以下方式支持:-> 1. **公益助学**:微信搜索**腾讯公益**,对里面的**助学**相关的项目随心捐。->-> 2. **赞助开发者**:你的赞助将用于补充碳基生物的咖啡因和硅基生物的 Token 消耗。---- **GitHub Issues**:适合针对性强的解答。提问时请提供完整信息(截图、错误日志、系统环境等)。-- **公众号交流**:适合快速咨询。建议优先在相关文章下的公共留言区交流,如私信,请文明礼貌用语😉-- **联系方式**:[email protected]---|公众号关注 |微信点赞 | 支付宝点赞 |-|:---:|:---:|:---:|-| <img src="_image/weixin.png" width="300" title="硅基茶水间"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F2ae0a88d98079f7e876c2b4dc85233c6-9e8025.JPG" width="300" title="微信支付"/> | <img src="https://cdn-1258574687.cos.ap-shanghai.myqcloud.com/img/%2F2025%2F07%2F17%2F1ed4f20ab8e35be51f8e84c94e6e239b4-fe4947.JPG" width="300" title="支付宝支付"/> |---<br>## 📚 项目相关diff --git a/_image/ai.jpg b/_image/ai.jpgnew file mode 100644index 0000000000..9167336585Binary files /dev/null and b/_image/ai.jpg differdiff --git a/_image/ai4.png b/_image/ai4.pngnew file mode 100644index 0000000000..39774ecce2Binary files /dev/null and b/_image/ai4.png differdiff --git a/config/ai_analysis_prompt.txt b/config/ai_analysis_prompt.txtindex 2024f6197a..766ac82f56 100644--- a/config/ai_analysis_prompt.txt+++ b/config/ai_analysis_prompt.txt@@ -3,96 +3,112 @@# ═══════════════════════════════════════════════════════════════## 此文件定义 AI 分析热点新闻时使用的提示词模板-# 你可以根据需要自定义分析角度和输出格式## 可用变量(在分析时会被替换):-# {report_mode} - 当前报告模式 (daily/current/incremental)+# {language} - 输出语言 (由 ai_analysis.language 配置)+# {report_mode} - 当前报告模式# {report_type} - 报告类型描述# {current_time} - 当前时间# {news_count} - 热榜新闻条数# {rss_count} - RSS 新闻条数# {keywords} - 匹配的关键词列表# {platforms} - 数据来源平台列表+# {news_content} - 热榜新闻内容+# {rss_content} - RSS 订阅内容 (需开启 ai_analysis.include_rss)## ═══════════════════════════════════════════════════════════════[system]你是一位专业的新闻分析师和趋势观察者。你的任务是分析热点新闻数据,提供有价值的洞察。-分析原则:-1. 客观中立 - 基于事实分析,避免主观臆断-2. 深度洞察 - 挖掘表面现象背后的趋势和规律-3. 实用价值 - 提供可操作的见解和建议-4. 简洁明了 - 用精炼的语言表达核心观点+## 核心原则-## 数据来源说明+1. 直击要害:避免废话,直接说"是什么"、"有多火"、"要注意什么"。+2. 逻辑闭环:将"现象"、"原因"与"建议"打通,告诉读者信息背后的行动指南。+3. 观点鲜明:明确指出是"泡沫"还是"机遇",是"争议"还是"共识"。+4. 通俗易懂:使用大众能理解的词汇(如"过热"、"降温"、"反转"、"出圈"),避免生造复杂概念。+5. 辩证思维:运用矛盾论视角,识别热点背后的"主要矛盾"与"次要矛盾",抓住事物发展的关键内因。-本系统从多个热榜平台(如微博、知乎、今日头条等)和 RSS 订阅源抓取新闻数据。-数据经过 frequency_words.txt 中定义的关键词过滤,只保留匹配的新闻。+## 数据字段深度解读指南-## 数据字段说明+为了做出精准判断,请充分利用以下数据维度:-### 热榜新闻字段-每条热榜新闻包含以下维度:-- 来源: 新闻所在的热榜平台(如微博热搜、知乎热榜、今日头条等)-- 标题: 新闻标题内容-- 排名: 该新闻在来源平台热榜中的排名范围,格式为"最高排名-最低排名"(如"1"表示排名稳定在第1,"3-8"表示最高冲到第3名、最低跌到第8名)-- 时间: 该新闻在热榜上出现的时间段,格式为"首次出现时间~最后出现时间"(如"09:30~12:45"表示从9:30首次上榜到12:45最后一次出现)-- 出现次数: 在监控时间段内,该新闻被抓取到的次数(次数越多说明在热榜上停留时间越长,热度越持久)+### 1. 基础维度+- 排名:"1"为榜首,数字越小越热。"3-8"表示排名在第3到第8之间波动。+- 出现次数:次数越多,说明在热榜由于停留时间越长,热度越持久。+- 时间范围:如"09:30~12:45",跨度越大说明话题生命力越强。-### RSS 新闻字段-每条 RSS 新闻包含:-- 来源: RSS 订阅源名称-- 标题: 文章标题-- 发布时间: 文章的原始发布时间+### 2. 轨迹量化分析 (重要)+当数据包含轨迹信息(如 `1(09:30)→0(10:00)→2(10:30)`)时,请关注:+- 急升/爆发:排名在短时间内大幅上升(如从20名升至3名),往往意味着重大突发事件。+- 僵尸热搜:排名持续阴跌且无反弹(如 10→15→20),说明热度正在衰退。+- 回榜/反转:脱榜(显示为0)后又重回高位,通常意味着有新爆料或反转剧情。-## 分析要点+### 3. 跨平台特征 (分级标准)+- 全网霸屏:5 个及以上平台同时上榜。真正的“国民级”话题,无死角覆盖。+- 破圈扩散:3-4 个平台同时上榜。话题已突破单一社区壁垒,正在向外蔓延。+- 圈层热点:仅在 1-2 个平台火爆。属于特定人群的狂欢(如仅在技术社区或娱乐榜)。-利用这些数据维度,你可以分析:-1. 热度强度: 排名越靠前(数字越小)、出现次数越多,热度越高-2. 持续时间: 时间跨度大、出现次数多,说明话题持续发酵-3. 排名波动: 排名范围大(如"1-20")说明热度不稳定,范围小(如"2-4")说明热度稳定-4. 跨平台热度: 同一话题在多个平台出现,说明影响力更广-5. 新兴趋势: 排名快速上升或首次出现的话题-6. 时效性: RSS 发布时间可判断信息新鲜度+## 分析板块说明 (5个核心板块)++1. 核心热点态势 (Core Trends & Momentum)+ - 整合:"趋势概述"、"热度走势"、"跨平台关联"。+ - 任务:直接定性当前最火的话题。结合排名和跨平台数据,判断是"全网刷屏"还是"圈层热议"。+ - 写法:避免简单罗列数据,而是总结态势。例如:"某话题霸榜多平台,热度持续超6小时,呈现极速爆发态势。"++2. 舆论风向争议 (Sentiment & Controversy)+ - 任务:运用矛盾分析法挖掘公众情绪内核。识别舆论场中的"根本对立"(主要矛盾)与"转化趋势",分析主流与非主流观点的博弈。+ - 重点:是否存在观点对立?(如技术乐观派 vs 隐私担忧派)。情绪是正面(期待、兴奋)、负面(愤怒、担忧)还是复杂(调侃、质疑)?++3. 异动与弱信号 (Signals)+ - 任务:通过"轨迹"和"排名变化"捕捉异常。+ - 关注:排名骤升的突发事件、首次出现的新鲜话题、或者反直觉的热度波动(如深夜突然高热)。++4. RSS 深度洞察 (RSS Insights)+ - 任务:分析 RSS 订阅源中的专业内容,提炼行业动态和深度信息。+ - 关注:技术博客的前沿观点、行业媒体的独家报道、与热榜话题的关联或差异。+ - 写法:突出 RSS 内容的"信息增量"——热榜没有但 RSS 有的独特视角或深度分析。++5. 研判策略建议 (Outlook & Strategy)+ - 整合:"潜在影响"与"建议"。+ - 任务:形成闭环。基于上述分析,预测后续走向(如"可能会引起监管注意"),并给出具体建议。+ - 对象:建议可面向投资者、品牌方或普通大众,力求落地。[user]请分析以下热点新闻数据:## 数据概览-- 报告模式:{report_mode}-- 报告类型:{report_type}+- 报告模式:{report_mode} ({report_type})- 分析时间:{current_time}-- 热榜新闻:{news_count} 条-- RSS 新闻:{rss_count} 条-- 数据来源:{platforms}+- 数据量:{news_count}条热榜 + {rss_count}条RSS+- 来源:{platforms}## 匹配关键词{keywords}-## 新闻内容+## 热榜新闻{news_content}+## RSS 订阅+{rss_content}+----请基于上述数据进行多维度分析,以 JSON 格式返回结果:+请基于上述数据撰写分析报告,以 JSON 格式返回结果:```json{- "summary": "核心热点概况(用简练语言概括当前最主要的核心事件,避免提及具体排名数据,80字以内)",- "keyword_analysis": "热度走势分析(结合排名波动、出现次数和时间跨度,分析核心话题的爆发力与持久性,80字以内)",- "sentiment": "情感倾向分析(极其重要:深入分析公众对核心话题的情感反馈,如:正面、负面、担忧、中性或争议,并简述原因,80字以内)",- "cross_platform": "跨平台联动分析(分析话题在多平台同步热搜的程度及其影响力差异,60字以内)",- "impact": "潜在影响评估(评估话题对社会舆论、行业动态或公众决策的冲击,60字以内)",- "signals": "异常与弱信号捕捉(关注排名骤升、首次出现或反直觉的波动,60字以内)",- "conclusion": "结论与建议(给出1-2条具有参考价值的操作性建议,40字以内)"+ "core_trends": "核心热点态势(200字以内)。语言要像"大白话"一样通俗,但要像"手术刀"一样精准。拒绝学术词汇。严格按以下格式分段(注意换行):\n(一句话直击本质的开场白)\n\n【宏观主线】:\n(用通俗的话概括大势,如:国外巨头忙基建,国内市场炒应用...)\n\n【微观领域】:\n1. (细分点1):(描述)\n2. (细分点2):(描述)",+ "sentiment_controversy": "舆论风向争议(100字以内)。先定性【整体】是褒是贬,再看【局部】有啥吵头。格式:\n【整体定性】:\n(如:全网都在骂,但也有人在这波流量里赚钱...)\n\n【争议焦点】:\n1. (焦点1):...\n2. (焦点2):...",+ "signals": "异动与弱信号(100字以内)。按信号类型分点:\n1. 急升信号:...\n2. 异动信号:...\n3. 弱信号:...",+ "rss_insights": "RSS 深度洞察(100字以内,无RSS数据时填"暂无RSS数据")。突出RSS的信息增量:\n【独家视角】:\n(热榜没有但RSS有的独特观点或深度分析)\n\n【行业动态】:\n(技术博客、行业媒体的前沿信息)",+ "outlook_strategy": "研判策略建议。分受众群体给出建议:\n1. 投资者:...\n2. 品牌方:...\n3. 公众:..."}```要求:- 必须返回有效的 JSON 格式-- 分析要结合排名、出现次数、时间跨度等数据维度-- 情感倾向分析是重点,请确保能够准确捕捉舆论风向-- 每个字段都要填写,如无明显发现可写"暂无明显特征"-- 使用中文-- 保持简洁,避免冗余内容在不同字段间重复+- 使用 {language} 输出,语言简练专业+- 确保 5 个板块不重叠,信息不冗余+- 若某板块无明显内容,可简写"暂无显著异常"+- 不要使用 Markdown 格式(如 **加粗**),仅使用纯文本diff --git a/config/ai_translation_prompt.txt b/config/ai_translation_prompt.txtnew file mode 100644index 0000000000..6f5d4895ec--- /dev/null+++ b/config/ai_translation_prompt.txt@@ -0,0 +1,27 @@+# ═══════════════════════════════════════════════════════════════+# TrendRadar AI 翻译提示词配置+# ═══════════════════════════════════════════════════════════════+#+# 此文件定义 AI 翻译内容时使用的提示词模板+#+# 可用变量:+# {target_language} - 目标语言+# {content} - 需要翻译的文本内容+#+# ═══════════════════════════════════════════════════════════════++[system]+你是一位精通多语言的专业翻译助手。你的任务是将新闻内容翻译成目标语言,保持新闻的专业性、准确性和简洁性。++要求:+1. 准确传达原文含义,不要遗漏关键信息。+2. 保持新闻标题的吸引力,但不要做标题党。+3. 专有名词(人名、地名、机构名)若有通用译名请使用通用译名,否则保留原文或在括号内备注。+4. 输出格式必须严格遵循要求,不要输出任何多余的解释性文字。++[user]+请将以下内容翻译成 {target_language}:++{content}++请直接输出翻译结果。diff --git a/config/config.yaml b/config/config.yamlindex 4652c1edb0..9441f66629 100644--- a/config/config.yaml+++ b/config/config.yaml@@ -20,32 +20,37 @@ app:# ===============================================================# 2. 数据源 - 热榜平台#-# id: 平台唯一标识(勿修改)-# name: 显示名称(可自定义,修改后不影响运行)+# enabled: 是否启用热榜抓取(总开关)+# sources: 平台列表+# - id: 平台唯一标识(勿修改)+# - name: 显示名称(可自定义,修改后不影响运行)# ===============================================================platforms:- - id: "toutiao"- name: "今日头条"- - id: "baidu"- name: "百度热搜"- - id: "wallstreetcn-hot"- name: "华尔街见闻"- - id: "thepaper"- name: "澎湃新闻"- - id: "bilibili-hot-search"- name: "bilibili 热搜"- - id: "cls-hot"- name: "财联社热门"- - id: "ifeng"- name: "凤凰网"- - id: "tieba"- name: "贴吧"- - id: "weibo"- name: "微博"- - id: "douyin"- name: "抖音"- - id: "zhihu"- name: "知乎"+ enabled: true # 是否启用热榜平台抓取+ sources:+ - id: "toutiao"+ name: "今日头条"+ - id: "baidu"+ name: "百度热搜"+ - id: "wallstreetcn-hot"+ name: "华尔街见闻"+ - id: "thepaper"+ name: "澎湃新闻"+ - id: "bilibili-hot-search"+ name: "bilibili 热搜"+ - id: "cls-hot"+ name: "财联社热门"+ - id: "ifeng"+ name: "凤凰网"+ - id: "tieba"+ name: "贴吧"+ - id: "weibo"+ name: "微博"+ - id: "douyin"+ name: "抖音"+ - id: "zhihu"+ name: "知乎"+# ===============================================================@@ -71,6 +76,7 @@ rss:# - 只有新鲜的文章会被推送到通知渠道freshness_filter:enabled: true # 是否启用新鲜度过滤(默认启用)+max_age_days: 3 # 最大文章年龄(天)# - 正整数:只推送 N 天内的文章# - 0:禁用过滤,推送所有文章@@ -123,17 +129,72 @@ rss:# ===============================================================report:mode: "current" # 可选: daily | current | incremental- display_mode: "keyword" # 可选: keyword | platform+ display_mode: "keyword" # 分组维度: keyword | platform# keyword: 按关键词分组显示(默认)# platform: 按平台/来源分组显示++ # 关键词组排序方式(仅 display_mode: keyword 时生效)+ # true: 按 frequency_words.txt 中的定义顺序排列+ # false: 按匹配到的热点条数排序(条数多的在前)+ sort_by_position_first: false+rank_threshold: 5 # 排名高亮阈值- sort_by_position_first: false # true=按配置位置排序,false=按热点条数排序+max_news_per_keyword: 0 # 每个关键词最大显示数量(0=不限制)- reverse_content_order: false # false=热点词汇统计在前,true=新增热点新闻在前# ===============================================================-# 5. 推送通知+# 5. 推送内容控制+#+# 统一管理推送消息中显示哪些区域及其排列顺序+# ===============================================================+display:+ # 📋 区域显示顺序+ # 列表从上到下的顺序 = 推送消息中从上到下的显示顺序+ # 想调整顺序?直接剪切粘贴整行即可,例如把 ai_analysis 移到最前面:+ # region_order:+ # - ai_analysis ← 移到第一行,AI 分析就会显示在最顶部+ # - new_items+ # - hotlist+ # - ...+ # 注意:区域需同时满足两个条件才会显示:+ # 1. 在此列表中+ # 2. 下方 regions 中对应开关为 true+ region_order:+ - new_items # 1️⃣ 新增热点区域+ - hotlist # 2️⃣ 热榜区域(关键词匹配)+ - rss # 3️⃣ RSS 订阅区域+ - standalone # 4️⃣ 独立展示区+ - ai_analysis # 5️⃣ AI 分析区域++ # 推送区域开关+ # 控制各区域是否启用(配合 region_order 使用)+ regions:+ hotlist: true # 热榜区域(关键词匹配的热点新闻)+ new_items: true # 新增热点区域(含热榜新增 + RSS 新增)+ # 注:热点词汇统计中的新增标记🆕不受此配置影响++ rss: true # RSS 订阅区域+ # 开启后将对 RSS 进行关键词分析并在通知中展示+ # 关闭后跳过分析,但独立展示区不受影响++ standalone: false # 独立展示区(完整热榜/RSS,不受关键词过滤)+ ai_analysis: true # AI 分析区域++ # 📋 独立展示区配置(仅在 regions.standalone: true 时生效)+ # 用途:将指定平台的完整热榜/RSS 单独展示,不受关键词过滤影响+ # 适用场景:+ # - 想完整查看某个平台的热榜排名+ # - RSS 源内容较少,希望全部展示而非只显示关键词匹配的+ # 注意:同一新闻可能同时出现在关键词匹配区和独立展示区+ standalone:+ platforms: [] # 热榜平台 ID 列表(如 ["zhihu", "weibo"])+ rss_feeds: [] # RSS 源 ID 列表(如 ["hacker-news"])+ max_items: 20 # 每个源最多展示条数(0=不限制)+++# ===============================================================+# 6. 推送通知## ⚠️ 重要安全警告 ⚠️#@@ -158,28 +219,17 @@ notification:# 🕐 推送时间窗口控制(可选功能)# 用途:限制推送的时间范围,避免非工作时间打扰# 适用场景:- # - 只想在工作日白天接收推送(如 09:00-18:00)- # - 希望在晚上固定时间收到汇总(如 20:00-22:00)- # 注意:GitHub Actions 执行时间不稳定,时间范围建议至少留足 2 小时- # 如果想要精准的定时推送,建议使用 Docker 部署在个人服务器上+ # • 只想在工作日白天接收推送(如 09:00-18:00)+ # • 希望在晚上固定时间收到汇总(如 20:00-22:00)+ # ⚠️ GitHub Actions 用户注意:+ # 执行时间不稳定,时间范围建议至少留足 2 小时+ # 💡 想要精准定时?建议使用 Docker 部署在个人服务器上push_window:enabled: false # 是否启用推送时间窗口控制start: "20:00" # 开始时间(北京时间)end: "22:00" # 结束时间(北京时间)once_per_day: true # true=窗口内只推送一次,false=窗口内每次执行都推送- # 📋 独立展示区配置(可选功能)- # 用途:将指定平台的完整热榜/RSS 单独展示,不受关键词过滤影响- # 适用场景:- # - 想完整查看某个平台的热榜排名- # - RSS 源内容较少,希望全部展示而非只显示关键词匹配的- # 注意:同一新闻可能同时出现在关键词匹配区和独立展示区- standalone_display:- enabled: false # 是否启用独立展示区- platforms: [] # 热榜平台 ID 列表(如 ["zhihu", "weibo"])- rss_feeds: [] # RSS 源 ID 列表(如 ["hacker-news"])- max_items: 20 # 每个源最多展示条数(0=不限制)-# 推送渠道配置channels:feishu:@@ -222,7 +272,7 @@ notification:# ===============================================================-# 6. 存储配置+# 7. 存储配置# ===============================================================storage:# 存储后端选择@@ -247,6 +297,7 @@ storage:# 建议将敏感信息配置在 GitHub Secrets 或环境变量中remote:retention_days: 0 # 保留天数(0=永久保留)+# S3 兼容配置(或使用环境变量 S3_ENDPOINT_URL 等)endpoint_url: "" # 服务端点# Cloudflare R2: https://<account_id>.r2.cloudflarestorage.com@@ -265,14 +316,12 @@ storage:# ===============================================================-# 7. AI 分析功能+# 8. AI 模型配置(共享)#-# 使用 AI 大模型对推送内容进行深度分析-# 支持 OpenAI、Anthropic、DeepSeek等兼容接口+# ai_analysis 和 ai_translation 共用此模型配置+# 支持 OpenAI、DeepSeek、Google Gemini 等兼容接口# ===============================================================-ai_analysis:- enabled: true # 是否启用 AI 分析-+ai:# AI 提供商配置# 支持的提供商:# - deepseek: DeepSeek(默认)@@ -294,39 +343,102 @@ ai_analysis:timeout: 90 # 请求超时(秒)- # 推送模式(仅在 enabled: true 时生效)- # - only_analysis: 仅推送 AI 分析结果(若开启了“独立展示区”则一并保留,屏蔽原始热榜/RSS 列表)- # - both: 两者都推送(分析追加在原始内容后)- # 注:如果不需要 AI 分析,请将上方 enabled 设为 false,无需使用 push_mode 控制- push_mode: "both"+ # AI 参数配置+ temperature: 1.0 # 采样温度 (0.0-2.0)+ # 注意:部分模型(如 gpt-5)可能要求必须为 1.0,否则会报错++ max_tokens: 5000 # 最大生成 token 数+ # 注意:如果 API 不支持此参数(报 HTTP 400),请设为 0 以禁用发送++ # ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━+ # 额外自定义参数 (高级选项)+ # ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━+ # 说明:用于向 AI 传递模型特定的高级生成参数。+ # ⚠️ 警告:如果你不了解这些参数的含义,强烈建议【不要改动】,保持当前的注释状态。+ # 填写了不符合模型要求的参数会导致 AI 分析报错并停止工作。+ #+ # 提示:不仅限于下方的示例,你可以根据模型 API 文档自行添加任何支持的字段。+ #+ # 操作:如果你确定需要修改,请删掉该行最前方的 "# " (井号和空格)。+ # 注意:如果这几行都带着井号,则代表不使用额外参数(最推荐做法)。+ # -------------------------------------------------------------+ # extra_params:+ # top_p: 1.0 # [通用] 核采样:值越小生成结果越集中+ # topK: 40 # [Gemini 专用] 限制候选词数量+ # presence_penalty: 0.0 # [OpenAI 专用] 鼓励模型谈论新话题+ # # 你也可以在此继续添加模型支持的其他新字段,例如 stop, logit_bias 等+++# ===============================================================+# 9. AI 分析功能+#+# 使用 AI 大模型对推送内容进行深度分析+# 模型配置见上方 ai 配置段+# ===============================================================+ai_analysis:+ enabled: true # 是否启用 AI 分析- # 分析选项+ # 分析报告输出语言+ # 格式:自然语言描述+ # 示例: "English", "Korean", "法语"+ language: "Chinese"++ # 提示词配置文件路径(相对于 config 目录)+ prompt_file: "ai_analysis_prompt.txt"++ # 分析内容配置max_news_for_analysis: 50 # 参与分析的新闻数量上限(控制成本关键项)+ # 当前默认的【报告模式】是【当前榜单模式】(current),也就是只分析当前在热榜的新闻+ # 如果需要让报告呈现出更有参考价值的完整一天的趋势,且你的 token 充裕+ # 可开启 daily(当日汇总模式)+ # 同时调整 max_news_for_analysis 为 150(你自己视情况调整,推送消息顶部有 ai 分析数目供参考)+# api 成本估算 (仅供参考)- # 按默认推送频率和模型- # GitHub Action 约 0.1 元/天- # Docker 部署约 0.2 元/天+ # 按默认推送频率和模型(deepseek)+ # 且 include_rank_timeline 为 false+ # 则+ # GitHub Action 部署默认推送约 20 次(每小时推送一次), 约 0.1 元/天+ # Docker 部署默认推送 48 次(每半小时推送一次), 约 0.2 元/天++ include_rss: false # 是否包含 RSS 内容进行分析++ include_rank_timeline: true # 是否传递完整排名时间线+ # false: 使用简化格式(排名范围+时间范围+出现次数)+ # true: 传递完整排名变化轨迹(如 1(09:30)→2(10:00)→0(11:00))+ # 启用后 AI 能更精确分析热度趋势,但会额外增加 token 消耗(0.5 倍到 1 倍)- include_rss: false # 是否包含 RSS 内容进行分析++# ===============================================================+# 10. AI 翻译功能+#+# 对推送内容进行多语言翻译,不包含 ai_analysis 分析的内容+# 模型配置见上方 ai 配置段+# ===============================================================+ai_translation:+ enabled: false # 是否启用翻译功能++ # 翻译目标语言+ # 格式:自然语言描述+ # 示例: "Chinese", "Korean", "法语"+ language: "English"# 提示词配置文件路径(相对于 config 目录)- prompt_file: "ai_analysis_prompt.txt"+ prompt_file: "ai_translation_prompt.txt"# ===============================================================-# 8. 高级设置(一般无需修改)+# 11. 高级设置(一般无需修改)# ===============================================================advanced:# 调试模式- debug: false+ debug: true# 版本检查version_check_url: "https://raw.githubusercontent.com/sansan0/TrendRadar/refs/heads/master/version"mcp_version_check_url: "https://raw.githubusercontent.com/sansan0/TrendRadar/refs/heads/master/version_mcp"- # 爬虫设置+ # 热榜爬虫技术参数crawler:- enabled: true # 是否启用爬取新闻功能request_interval: 2000 # 请求间隔(毫秒)use_proxy: false # 是否启用代理default_proxy: "http://127.0.0.1:10801"@@ -337,7 +449,6 @@ advanced:timeout: 15 # 请求超时(秒)use_proxy: false # 是否使用代理proxy_url: "" # RSS 专属代理(留空则使用 crawler.default_proxy)- notification_enabled: true # 是否启用 RSS 通知推送# 排序权重(用于重新排序不同平台的热搜)# 合起来等于 1diff --git a/config/frequency_words.txt b/config/frequency_words.txtindex 5ec6c28003..940d6e4d78 100644--- a/config/frequency_words.txt+++ b/config/frequency_words.txt@@ -1,6 +1,7 @@# ═══════════════════════════════════════════════════════════════# TrendRadar 频率词配置文件# ═══════════════════════════════════════════════════════════════+# 凡是左侧有 # 的都是仅供阅读的说明性文字## 这个文件用来设置你想关注的新闻关键词。# 系统会自动抓取包含这些关键词的热榜新闻推送给你。diff --git a/docker/.env b/docker/.envindex 69a6b8da94..8ae43ea4e3 100644--- a/docker/.env+++ b/docker/.env@@ -1,16 +1,3 @@-# ============================================-# 核心配置(环境变量优先级 > config.yaml)-# ============================================--# 是否启用爬虫 (true/false)-ENABLE_CRAWLER=-# 是否启用通知 (true/false)-ENABLE_NOTIFICATION=-# 报告模式 (daily|incremental|current)-REPORT_MODE=-# 显示模式 (keyword|platform)-DISPLAY_MODE=-# ============================================# Web 服务器配置# ============================================@@ -70,12 +57,12 @@ GENERIC_WEBHOOK_URL=GENERIC_WEBHOOK_TEMPLATE=# ============================================-# AI 分析配置+# AI 配置(ai_analysis 和 ai_translation 共享模型配置)# ============================================# 是否启用 AI 分析 (true/false)AI_ANALYSIS_ENABLED=false-# AI API Key(必填,启用 AI 分析时需要)+# AI API Key(必填,启用 AI 功能时需要)AI_API_KEY=# AI 提供商 (deepseek|openai|gemini|custom)AI_PROVIDER=deepseekdiff --git a/docker/docker-compose-build.yml b/docker/docker-compose-build.ymlindex 461953822f..54c5dd9735 100644--- a/docker/docker-compose-build.yml+++ b/docker/docker-compose-build.yml@@ -15,11 +15,6 @@ services:environment:- TZ=Asia/Shanghai- # 核心配置- - ENABLE_CRAWLER=${ENABLE_CRAWLER:-}- - ENABLE_NOTIFICATION=${ENABLE_NOTIFICATION:-}- - REPORT_MODE=${REPORT_MODE:-}- - DISPLAY_MODE=${DISPLAY_MODE:-}# Web 服务器- ENABLE_WEBSERVER=${ENABLE_WEBSERVER:-false}- WEBSERVER_PORT=${WEBSERVER_PORT:-8080}@@ -47,7 +42,7 @@ services:# 通用Webhook配置- GENERIC_WEBHOOK_URL=${GENERIC_WEBHOOK_URL:-}- GENERIC_WEBHOOK_TEMPLATE=${GENERIC_WEBHOOK_TEMPLATE:-}- # AI 分析配置+ # AI 配置(ai_analysis 和 ai_translation 共享模型配置)- AI_ANALYSIS_ENABLED=${AI_ANALYSIS_ENABLED:-false}- AI_API_KEY=${AI_API_KEY:-}- AI_PROVIDER=${AI_PROVIDER:-}diff --git a/docker/docker-compose.yml b/docker/docker-compose.ymlindex dc9b16fde7..b1b37af7f5 100644--- a/docker/docker-compose.yml+++ b/docker/docker-compose.yml@@ -13,11 +13,6 @@ services:environment:- TZ=Asia/Shanghai- # 核心配置- - ENABLE_CRAWLER=${ENABLE_CRAWLER:-}- - ENABLE_NOTIFICATION=${ENABLE_NOTIFICATION:-}- - REPORT_MODE=${REPORT_MODE:-}- - DISPLAY_MODE=${DISPLAY_MODE:-}# Web 服务器- ENABLE_WEBSERVER=${ENABLE_WEBSERVER:-false}- WEBSERVER_PORT=${WEBSERVER_PORT:-8080}@@ -45,7 +40,7 @@ services:# 通用Webhook配置- GENERIC_WEBHOOK_URL=${GENERIC_WEBHOOK_URL:-}- GENERIC_WEBHOOK_TEMPLATE=${GENERIC_WEBHOOK_TEMPLATE:-}- # AI 分析配置+ # AI 配置(ai_analysis 和 ai_translation 共享模型配置)- AI_ANALYSIS_ENABLED=${AI_ANALYSIS_ENABLED:-false}- AI_API_KEY=${AI_API_KEY:-}- AI_PROVIDER=${AI_PROVIDER:-}diff --git a/docker/manage.py b/docker/manage.pyindex e59cef940a..073bf21930 100644--- a/docker/manage.py+++ b/docker/manage.py@@ -279,11 +279,6 @@ def show_config():"CRON_SCHEDULE","RUN_MODE","IMMEDIATE_RUN",- # 核心配置- "ENABLE_CRAWLER",- "ENABLE_NOTIFICATION",- "REPORT_MODE",- "DISPLAY_MODE",# 通知渠道"FEISHU_WEBHOOK_URL","DINGTALK_WEBHOOK_URL",diff --git a/mcp_server/__init__.py b/mcp_server/__init__.pyindex 045c1dcb69..92d3af21dd 100644--- a/mcp_server/__init__.py+++ b/mcp_server/__init__.py@@ -5,4 +5,4 @@ TrendRadar MCP Server"""-__version__ = "3.1.5"+__version__ = "3.1.6"diff --git a/mcp_server/server.py b/mcp_server/server.pyindex 6671cb9559..366ff35fcf 100644--- a/mcp_server/server.py+++ b/mcp_server/server.py@@ -9,8 +9,7 @@ import asyncioimport jsonfrom typing import List, Optional, Dict, Union-from fastmcp import FastMCP, Context-from fastmcp.server.dependencies import get_context+from fastmcp import FastMCPfrom .tools.data_query import DataQueryToolsfrom .tools.analytics import AnalyticsTools@@ -28,9 +27,6 @@ mcp = FastMCP('trendradar-news')# 全局工具实例(在第一次请求时初始化)_tools_instances = {}-# Session-level 工具实例存储(用于 Context 管理)-_session_tools: Dict[str, Dict] = {}-def _get_tools(project_root: Optional[str] = None):"""获取或创建工具实例(单例模式)"""@@ -44,39 +40,6 @@ def _get_tools(project_root: Optional[str] = None):return _tools_instances-def _get_tools_with_context(ctx: Optional[Context] = None) -> Dict:- """- 获取工具实例(支持 Session 隔离)-- 如果提供了 Context,则为每个 session 创建独立的工具实例。- 这样可以避免不同会话之间的状态污染。-- Args:- ctx: FastMCP Context 对象-- Returns:- 工具实例字典- """- if ctx is None:- return _get_tools()-- # 获取 session ID(如果有的话)- session_id = getattr(ctx, 'session_id', None) or 'default'-- if session_id not in _session_tools:- # 为新 session 创建工具实例- _session_tools[session_id] = {- 'data': DataQueryTools(),- 'analytics': AnalyticsTools(),- 'search': SearchTools(),- 'config': ConfigManagementTools(),- 'system': SystemManagementTools(),- 'storage': StorageSyncTools(),- }-- return _session_tools[session_id]--# ==================== MCP Resources ====================@mcp.resource("config://platforms")@@ -229,28 +192,17 @@ async def get_latest_news(获取最新一批爬取的新闻数据,快速了解当前热点Args:- platforms: 平台ID列表,如 ['zhihu', 'weibo', 'douyin']- - 不指定时:使用 config.yaml 中配置的所有平台- - 支持的平台来自 config/config.yaml 的 platforms 配置- - 每个平台都有对应的name字段(如"知乎"、"微博"),方便AI识别+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台limit: 返回条数限制,默认50,最大1000- 注意:实际返回数量可能少于请求值,取决于当前可用的新闻总数include_url: 是否包含URL链接,默认False(节省token)Returns:JSON格式的新闻列表- **重要:数据展示建议**- 本工具会返回完整的新闻列表(通常50条)给你。但请注意:- - **工具返回**:完整的50条数据 ✅- - **建议展示**:向用户展示全部数据,除非用户明确要求总结- - **用户期望**:用户可能需要完整数据,请谨慎总结-- **何时可以总结**:- - 用户明确说"给我总结一下"或"挑重点说"- - 数据量超过100条时,可先展示部分并询问是否查看全部-- **注意**:如果用户询问"为什么只显示了部分",说明他们需要完整数据+ **数据展示建议**+ - 默认展示全部返回数据,除非用户明确要求总结+ - 用户说"总结"或"挑重点"时才进行筛选+ - 用户问"为什么只显示部分"说明需要完整数据"""tools = _get_tools()result = await asyncio.to_thread(@@ -308,35 +260,17 @@ async def get_latest_rss(RSS 数据与热榜新闻分开存储,按时间流展示,适合获取特定来源的最新内容。Args:- feeds: RSS 源 ID 列表,如 ['hacker-news', '36kr']- - 不指定时:返回所有已配置 RSS 源的数据- - 支持的 RSS 源来自 config/config.yaml 的 rss.feeds 配置+ feeds: RSS 源 ID 列表,如 ['hacker-news', '36kr'],不指定则返回所有源days: 获取最近 N 天的数据,默认 1(仅今天),最大 30 天- - 1: 仅今天(默认)- - 7: 最近一周- - 30: 最近一个月limit: 返回条数限制,默认50,最大500include_summary: 是否包含文章摘要,默认False(节省token)Returns:- JSON格式的 RSS 条目列表,包含:- - rss: RSS 条目数组- - title: 文章标题- - feed_id: RSS 源 ID- - feed_name: RSS 源名称- - url: 文章链接- - published_at: 发布时间- - author: 作者(如有)- - date: 数据日期- - summary: 摘要(仅当 include_summary=True)- - total: 返回条数- - feeds: 请求的 RSS 源列表+ JSON格式的 RSS 条目列表Examples:- - 获取今天所有 RSS: get_latest_rss()- - 获取最近一周: get_latest_rss(days=7)- - 获取指定源最近7天: get_latest_rss(feeds=['hacker-news'], days=7)- - 包含摘要: get_latest_rss(include_summary=True, days=7, limit=20)+ - get_latest_rss()+ - get_latest_rss(days=7, feeds=['hacker-news'])"""tools = _get_tools()result = await asyncio.to_thread(@@ -425,28 +359,12 @@ async def get_news_by_date(- 自然语言: "今天", "昨天", "本周", "最近7天"- 单日字符串: "2025-01-15"- 默认值: "今天"- platforms: 平台ID列表,如 ['zhihu', 'weibo', 'douyin']- - 不指定时:使用 config.yaml 中配置的所有平台- - 支持的平台来自 config/config.yaml 的 platforms 配置- - 每个平台都有对应的name字段(如"知乎"、"微博"),方便AI识别+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台limit: 返回条数限制,默认50,最大1000- 注意:实际返回数量可能少于请求值,取决于指定日期的新闻总数include_url: 是否包含URL链接,默认False(节省token)Returns:JSON格式的新闻列表,包含标题、平台、排名等信息-- **重要:数据展示建议**- 本工具会返回完整的新闻列表(通常50条)给你。但请注意:- - **工具返回**:完整的50条数据 ✅- - **建议展示**:向用户展示全部数据,除非用户明确要求总结- - **用户期望**:用户可能需要完整数据,请谨慎总结-- **何时可以总结**:- - 用户明确说"给我总结一下"或"挑重点说"- - 数据量超过100条时,可先展示部分并询问是否查看全部-- **注意**:如果用户询问"为什么只显示了部分",说明他们需要完整数据"""tools = _get_tools()result = await asyncio.to_thread(@@ -476,23 +394,17 @@ async def analyze_topic_trend("""统一话题趋势分析工具 - 整合多种趋势分析模式- **重要:日期范围处理**- 当用户使用"本周"、"最近7天"等自然语言时,请先调用 resolve_date_range 工具获取精确日期:- 1. 调用 resolve_date_range("本周") → 获取 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- 2. 将返回的 date_range 传入本工具+ 建议:使用自然语言日期时,先调用 resolve_date_range 获取精确日期范围。Args:topic: 话题关键词(必需)- analysis_type: 分析类型,可选值:- - "trend": 热度趋势分析(追踪话题的热度变化)- - "lifecycle": 生命周期分析(从出现到消失的完整周期)- - "viral": 异常热度检测(识别突然爆火的话题)- - "predict": 话题预测(预测未来可能的热点)- date_range: 日期范围(trend和lifecycle模式),可选- - **格式**: {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- - **获取方式**: 调用 resolve_date_range 工具解析自然语言日期- - **默认**: 不指定时默认分析最近7天- granularity: 时间粒度(trend模式),默认"day"(仅支持 day,因为底层数据按天聚合)+ analysis_type: 分析类型+ - "trend": 热度趋势分析(默认)+ - "lifecycle": 生命周期分析+ - "viral": 异常热度检测+ - "predict": 话题预测+ date_range: 日期范围,格式 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"},默认最近7天+ granularity: 时间粒度,默认"day"spike_threshold: 热度突增倍数阈值(viral模式),默认3.0time_window: 检测时间窗口小时数(viral模式),默认24lookahead_hours: 预测未来小时数(predict模式),默认6@@ -502,15 +414,8 @@ async def analyze_topic_trend(JSON格式的趋势分析结果Examples:- 用户:"分析AI本周的趋势"- 推荐调用流程:- 1. resolve_date_range("本周") → {"date_range": {"start": "2025-11-18", "end": "2025-11-26"}}- 2. analyze_topic_trend(topic="AI", date_range={"start": "2025-11-18", "end": "2025-11-26"})-- 用户:"看看特斯拉最近30天的热度"- 推荐调用流程:- 1. resolve_date_range("最近30天") → {"date_range": {"start": "2025-10-28", "end": "2025-11-26"}}- 2. analyze_topic_trend(topic="特斯拉", analysis_type="lifecycle", date_range=...)+ - analyze_topic_trend(topic="AI", date_range={"start": "2025-01-01", "end": "2025-01-07"})+ - analyze_topic_trend(topic="特斯拉", analysis_type="lifecycle")"""tools = _get_tools()result = await asyncio.to_thread(@@ -583,24 +488,13 @@ async def analyze_sentiment("""分析新闻的情感倾向和热度趋势- **重要:日期范围处理**- 当用户使用"本周"、"最近7天"等自然语言时,请先调用 resolve_date_range 工具获取精确日期:- 1. 调用 resolve_date_range("本周") → 获取 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- 2. 将返回的 date_range 传入本工具+ 建议:使用自然语言日期时,先调用 resolve_date_range 获取精确日期范围。Args:topic: 话题关键词(可选)- platforms: 平台ID列表,如 ['zhihu', 'weibo', 'douyin']- - 不指定时:使用 config.yaml 中配置的所有平台- - 支持的平台来自 config/config.yaml 的 platforms 配置- - 每个平台都有对应的name字段(如"知乎"、"微博"),方便AI识别- date_range: 日期范围(可选)- - **格式**: {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- - **获取方式**: 调用 resolve_date_range 工具解析自然语言日期- - **默认**: 不指定则默认查询今天的数据- limit: 返回新闻数量,默认50,最大100- 注意:本工具会对新闻标题进行去重(同一标题在不同平台只保留一次),- 因此实际返回数量可能少于请求的 limit 值+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台+ date_range: 日期范围,格式 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"},默认今天+ limit: 返回新闻数量,默认50,最大100(会对标题去重)sort_by_weight: 是否按热度权重排序,默认Trueinclude_url: 是否包含URL链接,默认False(节省token)@@ -608,20 +502,7 @@ async def analyze_sentiment(JSON格式的分析结果,包含情感分布、热度趋势和相关新闻Examples:- 用户:"分析AI本周的情感倾向"- 推荐调用流程:- 1. resolve_date_range("本周") → {"date_range": {"start": "2025-11-18", "end": "2025-11-26"}}- 2. analyze_sentiment(topic="AI", date_range={"start": "2025-11-18", "end": "2025-11-26"})-- 用户:"分析特斯拉最近7天的新闻情感"- 推荐调用流程:- 1. resolve_date_range("最近7天") → {"date_range": {"start": "2025-11-20", "end": "2025-11-26"}}- 2. analyze_sentiment(topic="特斯拉", date_range={"start": "2025-11-20", "end": "2025-11-26"})-- **重要:数据展示策略**- - 本工具返回完整的分析结果和新闻列表- - **默认展示方式**:展示完整的分析结果(包括所有新闻)- - 仅在用户明确要求"总结"或"挑重点"时才进行筛选+ - analyze_sentiment(topic="AI", date_range={"start": "2025-01-01", "end": "2025-01-07"})"""tools = _get_tools()result = await asyncio.to_thread(@@ -651,13 +532,9 @@ async def find_related_news(reference_title: 参考新闻标题(完整或部分)date_range: 日期范围(可选)- 不指定: 只查询今天的数据- - "today": 今天- - "yesterday": 昨天- - "last_week": 最近7天- - "last_month": 最近30天+ - "today", "yesterday", "last_week", "last_month": 预设值- {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}: 自定义范围- threshold: 相似度阈值,0-1之间,默认0.5- 注意:阈值越高匹配越严格,返回结果越少+ threshold: 相似度阈值,0-1之间,默认0.5(越高匹配越严格)limit: 返回条数限制,默认50include_url: 是否包含URL链接,默认False(节省token)@@ -665,13 +542,8 @@ async def find_related_news(JSON格式的相关新闻列表,按相似度排序Examples:- - 查找今天的相似新闻: find_related_news(reference_title="特斯拉降价")- - 查找历史相关新闻: find_related_news(reference_title="特斯拉降价", date_range="last_week")- - 自定义日期范围: find_related_news(reference_title="AI突破", date_range={"start": "2025-01-01", "end": "2025-01-15"})-- **重要:数据展示策略**- - 本工具返回完整的相关新闻列表(包括相似度分数)- - 仅在用户明确要求"总结"时才进行筛选+ - find_related_news(reference_title="特斯拉降价")+ - find_related_news(reference_title="AI突破", date_range="last_week")"""tools = _get_tools()result = await asyncio.to_thread(@@ -723,46 +595,21 @@ async def aggregate_news("""跨平台新闻聚合 - 对相似新闻进行去重合并- 将不同平台报道的同一事件合并为一条聚合新闻,- 显示该新闻在各平台的覆盖情况和综合热度。-- **使用场景:**- - 想要看到去重后的热点新闻(避免同一事件在不同平台重复展示)- - 分析某个话题在多个平台的覆盖情况- - 获取跨平台的综合热度排名+ 将不同平台报道的同一事件合并为一条聚合新闻,显示跨平台覆盖情况和综合热度。Args:- date_range: 日期范围(可选)- - 不指定: 查询今天- - {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}: 日期范围- platforms: 平台过滤列表,如 ['zhihu', 'weibo']- similarity_threshold: 相似度阈值,0.3-1.0之间,默认0.7- 越高越严格(仅合并非常相似的标题)+ date_range: 日期范围,不指定则查询今天+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台+ similarity_threshold: 相似度阈值,0.3-1.0,默认0.7(越高越严格)limit: 返回聚合新闻数量,默认50include_url: 是否包含URL链接,默认FalseReturns:- JSON格式的聚合结果,包含:- - summary: 聚合统计(原始数量、去重后数量、去重率)- - aggregated_news: 聚合后的新闻列表- - representative_title: 代表标题- - platforms: 覆盖的平台列表- - platform_count: 覆盖平台数- - is_cross_platform: 是否跨平台新闻- - best_rank: 最佳排名- - aggregate_weight: 综合权重- - sources: 各平台来源详情- - statistics: 平台覆盖统计+ JSON格式的聚合结果,包含去重统计、聚合新闻列表和平台覆盖统计Examples:- - aggregate_news() # 聚合今天所有平台的新闻- - aggregate_news(similarity_threshold=0.8) # 更严格的相似度匹配- - aggregate_news(date_range={"start": "2025-01-01", "end": "2025-01-07"})-- **重要:数据展示策略**- - 本工具返回去重聚合后的新闻列表- - 跨平台新闻(is_cross_platform=true)通常更具新闻价值- - 可优先展示 platform_count > 1 的新闻+ - aggregate_news()+ - aggregate_news(similarity_threshold=0.8)"""tools = _get_tools()result = await asyncio.to_thread(@@ -854,67 +701,30 @@ async def search_news("""统一搜索接口,支持多种搜索模式,可同时搜索热榜和RSS- **重要:日期范围处理**- 当用户使用"本周"、"最近7天"等自然语言时,请先调用 resolve_date_range 工具获取精确日期:- 1. 调用 resolve_date_range("本周") → 获取 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- 2. 将返回的 date_range 传入本工具+ 建议:使用自然语言日期时,先调用 resolve_date_range 获取精确日期范围。Args:query: 搜索关键词或内容片段- search_mode: 搜索模式,可选值:- - "keyword": 精确关键词匹配(默认,适合搜索特定话题)- - "fuzzy": 模糊内容匹配(适合搜索内容片段,会过滤相似度低于阈值的结果)- - "entity": 实体名称搜索(适合搜索人物/地点/机构)- date_range: 日期范围(可选)- - **格式**: {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}- - **获取方式**: 调用 resolve_date_range 工具解析自然语言日期- - **默认**: 不指定时默认查询今天的新闻- platforms: 平台ID列表,如 ['zhihu', 'weibo', 'douyin']- - 不指定时:使用 config.yaml 中配置的所有平台- - 支持的平台来自 config/config.yaml 的 platforms 配置- - 每个平台都有对应的name字段(如"知乎"、"微博"),方便AI识别- limit: 热榜返回条数限制,默认50,最大1000- 注意:实际返回数量取决于搜索匹配结果(特别是 fuzzy 模式下会过滤低相似度结果)- sort_by: 排序方式,可选值:- - "relevance": 按相关度排序(默认)- - "weight": 按新闻权重排序- - "date": 按日期排序- threshold: 相似度阈值(仅fuzzy模式有效),0-1之间,默认0.6- 注意:阈值越高匹配越严格,返回结果越少- include_url: 是否包含URL链接,默认False(节省token)- include_rss: 是否同时搜索RSS订阅数据,默认False- - 设为True时,会在热榜结果后附加RSS搜索结果- - RSS结果独立展示,不影响热榜排名- rss_limit: RSS返回条数限制,默认20(仅当include_rss=True时有效)+ search_mode: 搜索模式+ - "keyword": 精确关键词匹配(默认)+ - "fuzzy": 模糊内容匹配+ - "entity": 实体名称搜索(人物/地点/机构)+ date_range: 日期范围,格式 {"start": "YYYY-MM-DD", "end": "YYYY-MM-DD"},默认今天+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台+ limit: 热榜返回条数限制,默认50+ sort_by: 排序方式 - "relevance"(相关度)/ "weight"(权重)/ "date"(日期)+ threshold: 相似度阈值(仅fuzzy模式),0-1,默认0.6+ include_url: 是否包含URL链接,默认False+ include_rss: 是否同时搜索RSS数据,默认False+ rss_limit: RSS返回条数限制,默认20Returns:- JSON格式的搜索结果,包含:- - results: 热榜新闻列表(按排名/相关度排序)- - rss: RSS订阅结果列表(仅当include_rss=True时返回)- - summary: 搜索统计信息+ JSON格式的搜索结果,包含热榜新闻列表和可选的RSS结果Examples:- 用户:"搜索本周的AI新闻"- 推荐调用流程:- 1. resolve_date_range("本周") → {"date_range": {"start": "2025-11-18", "end": "2025-11-26"}}- 2. search_news(query="AI", date_range={"start": "2025-11-18", "end": "2025-11-26"})-- 用户:"搜索AI相关内容,包括RSS"- → search_news(query="AI", include_rss=True)-- 用户:"最近7天的特斯拉新闻"- 推荐调用流程:- 1. resolve_date_range("最近7天") → {"date_range": {"start": "2025-11-20", "end": "2025-11-26"}}- 2. search_news(query="特斯拉", date_range={"start": "2025-11-20", "end": "2025-11-26"})-- 用户:"今天的AI新闻"(默认今天,无需解析)- → search_news(query="AI")-- **重要:数据展示策略**- - 本工具返回完整的搜索结果列表- - **默认展示方式**:展示全部返回的新闻,无需总结或筛选- - 仅在用户明确要求"总结"或"挑重点"时才进行筛选- - 当include_rss=True时,热榜和RSS结果分开展示,RSS在热榜之后+ - search_news(query="AI")+ - search_news(query="AI", include_rss=True)+ - search_news(query="特斯拉", date_range={"start": "2025-01-01", "end": "2025-01-07"})"""tools = _get_tools()result = await asyncio.to_thread(@@ -981,37 +791,16 @@ async def check_version(检查版本更新(同时检查 TrendRadar 和 MCP Server)比较本地版本与 GitHub 远程版本,判断是否需要更新。- 远程版本 URL 从 config.yaml 获取:- - version_check_url: TrendRadar 版本- - mcp_version_check_url: MCP Server 版本Args:proxy_url: 可选的代理URL,用于访问 GitHub(如 http://127.0.0.1:7890)Returns:- JSON格式的版本检查结果,包含:- - success: 是否成功- - summary:- - description: 结果描述- - any_update: 是否有任何组件需要更新- - data:- - trendradar: TrendRadar 版本检查结果- - name: 组件名称- - current_version: 当前本地版本(如 "5.0.0")- - remote_version: 远程最新版本- - need_update: 是否需要更新- - message: 状态描述- - mcp: MCP Server 版本检查结果- - name: 组件名称- - current_version: 当前本地版本(如 "3.1.4")- - remote_version: 远程最新版本- - need_update: 是否需要更新- - message: 状态描述- - any_update: 是否有任何组件需要更新+ JSON格式的版本检查结果,包含两个组件的版本对比和是否需要更新Examples:- - check_version() # 直接检查两个组件的版本- - check_version(proxy_url="http://127.0.0.1:7890") # 使用代理访问 GitHub+ - check_version()+ - check_version(proxy_url="http://127.0.0.1:7890")"""tools = _get_tools()result = await asyncio.to_thread(tools['system'].check_version, proxy_url=proxy_url)@@ -1028,25 +817,16 @@ async def trigger_crawl(手动触发一次爬取任务(可选持久化)Args:- platforms: 指定平台ID列表,如 ['zhihu', 'weibo', 'douyin']- - 不指定时:使用 config.yaml 中配置的所有平台- - 支持的平台来自 config/config.yaml 的 platforms 配置- - 每个平台都有对应的name字段(如"知乎"、"微博"),方便AI识别- - 注意:失败的平台会在返回结果的 failed_platforms 字段中列出+ platforms: 平台ID列表,如 ['zhihu', 'weibo'],不指定则使用所有平台save_to_local: 是否保存到本地 output 目录,默认 Falseinclude_url: 是否包含URL链接,默认False(节省token)Returns:- JSON格式的任务状态信息,包含:- - platforms: 成功爬取的平台列表- - failed_platforms: 失败的平台列表(如有)- - total_news: 爬取的新闻总数- - data: 新闻数据+ JSON格式的任务状态信息,包含成功/失败平台列表和新闻数据Examples:- - 临时爬取: trigger_crawl(platforms=['zhihu'])- - 爬取并保存: trigger_crawl(platforms=['weibo'], save_to_local=True)- - 使用默认平台: trigger_crawl() # 爬取config.yaml中配置的所有平台+ - trigger_crawl(platforms=['zhihu'])+ - trigger_crawl(save_to_local=True)"""tools = _get_tools()result = await asyncio.to_thread(@@ -1107,26 +887,7 @@ async def get_storage_status() -> str:查看当前存储后端配置、本地和远程存储的状态信息。Returns:- JSON格式的存储状态信息,包含:- - backend: 当前使用的后端类型(local/remote/auto)- - local: 本地存储状态- - data_dir: 数据目录- - retention_days: 保留天数- - total_size: 总大小- - date_count: 日期数量- - earliest_date: 最早日期- - latest_date: 最新日期- - remote: 远程存储状态- - configured: 是否已配置- - endpoint_url: 服务端点- - bucket_name: 存储桶名称- - date_count: 远程日期数量- - pull: 拉取配置- - enabled: 是否启用自动拉取- - days: 自动拉取天数-- Examples:- - get_storage_status() # 查看所有存储状态+ JSON格式的存储状态信息,包含本地/远程存储状态和拉取配置"""tools = _get_tools()result = await asyncio.to_thread(tools['storage'].get_storage_status)@@ -1140,37 +901,20 @@ async def list_available_dates("""列出本地/远程可用的日期范围- 查看本地和远程存储中有哪些日期的数据可用,- 帮助了解数据覆盖范围和同步状态。+ 查看本地和远程存储中有哪些日期的数据可用。Args:- source: 数据来源,可选值:- - "local": 仅列出本地可用日期- - "remote": 仅列出远程可用日期- - "both": 同时列出两者并进行对比(默认)+ source: 数据来源+ - "local": 仅本地+ - "remote": 仅远程+ - "both": 同时列出并对比(默认)Returns:- JSON格式的日期列表,包含:- - local: 本地日期信息(如果 source 包含 local)- - dates: 日期列表(按时间倒序)- - count: 日期数量- - earliest: 最早日期- - latest: 最新日期- - remote: 远程日期信息(如果 source 包含 remote)- - configured: 是否已配置远程存储- - dates: 日期列表- - count: 日期数量- - earliest: 最早日期- - latest: 最新日期- - comparison: 对比结果(仅当 source="both" 时)- - only_local: 仅本地存在的日期- - only_remote: 仅远程存在的日期- - both: 两边都存在的日期+ JSON格式的日期列表,包含各来源的日期信息和对比结果Examples:- - list_available_dates() # 查看本地和远程的对比- - list_available_dates(source="local") # 仅查看本地- - list_available_dates(source="remote") # 仅查看远程+ - list_available_dates()+ - list_available_dates(source="local")"""tools = _get_tools()result = await asyncio.to_thread(tools['storage'].list_available_dates, source=source)diff --git a/mcp_server/services/data_service.py b/mcp_server/services/data_service.pyindex b12ddde90d..0012dda0fe 100644--- a/mcp_server/services/data_service.py+++ b/mcp_server/services/data_service.py@@ -150,7 +150,7 @@ class DataService:# 尝试从缓存获取date_str = target_date.strftime("%Y-%m-%d")cache_key = f"news_by_date:{date_str}:{','.join(platforms or [])}:{limit}:{include_url}"- cached = self.cache.get(cache_key, ttl=1800) # 30分钟缓存+ cached = self.cache.get(cache_key, ttl=900) # 15分钟缓存if cached:return cached@@ -353,7 +353,7 @@ class DataService:"""# 尝试从缓存获取cache_key = f"trending_topics:{top_n}:{mode}:{extract_mode}"- cached = self.cache.get(cache_key, ttl=1800) # 30分钟缓存+ cached = self.cache.get(cache_key, ttl=900) # 15分钟缓存if cached:return cached@@ -470,12 +470,6 @@ class DataService:Raises:FileParseError: 配置文件解析错误"""- # 尝试从缓存获取- cache_key = f"config:{section}"- cached = self.cache.get(cache_key, ttl=3600) # 1小时缓存- if cached:- return cached-# 解析配置文件config_data = self.parser.parse_yaml_config()word_groups = self.parser.parse_frequency_words()@@ -483,14 +477,15 @@ class DataService:# 根据section返回对应配置advanced = config_data.get("advanced", {})advanced_crawler = advanced.get("crawler", {})+ platforms_config = config_data.get("platforms", {})if section == "all" or section == "crawler":crawler_config = {- "enable_crawler": advanced_crawler.get("enabled", True),+ "enable_crawler": platforms_config.get("enabled", True),"use_proxy": advanced_crawler.get("use_proxy", False),"request_interval": advanced_crawler.get("request_interval", 1),"retry_times": 3,- "platforms": [p["id"] for p in config_data.get("platforms", [])]+ "platforms": [p["id"] for p in platforms_config.get("sources", [])]}if section == "all" or section == "push":@@ -545,9 +540,6 @@ class DataService:else:result = {}- # 缓存结果- self.cache.set(cache_key, result)-return resultdef get_available_date_range(self) -> Tuple[Optional[datetime], Optional[datetime]]:@@ -863,7 +855,7 @@ class DataService:RSS 源状态信息"""cache_key = "rss_feeds_status"- cached = self.cache.get(cache_key, ttl=300)+ cached = self.cache.get(cache_key, ttl=900)if cached:return cacheddiff --git a/mcp_server/services/parser_service.py b/mcp_server/services/parser_service.pyindex 01250f2211..5fe7f0b868 100644--- a/mcp_server/services/parser_service.py+++ b/mcp_server/services/parser_service.py@@ -325,7 +325,7 @@ class ParserService:cache_key = f"read_all:{db_type}:{date_str}:{platform_key}"is_today = (date is None) or (date.date() == datetime.now().date())- ttl = 900 if is_today else 3600+ ttl = 900 if is_today else 900cached = self.cache.get(cache_key, ttl=ttl)if cached:diff --git a/mcp_server/tools/config_mgmt.py b/mcp_server/tools/config_mgmt.pyindex 25ab7f5d98..306ffab428 100644--- a/mcp_server/tools/config_mgmt.py+++ b/mcp_server/tools/config_mgmt.py@@ -4,13 +4,28 @@实现配置查询和管理功能。"""-from typing import Dict, Optional+from typing import Dict, Optional, Any, TypedDictfrom ..services.data_service import DataServicefrom ..utils.validators import validate_config_sectionfrom ..utils.errors import MCPError+class ErrorInfo(TypedDict, total=False):+ """错误信息结构"""+ code: str+ message: str+ suggestion: str+++class ConfigResult(TypedDict):+ """配置查询结果 - success 字段必需,其他字段可选"""+ success: bool+ config: Optional[Dict[str, Any]]+ section: Optional[str]+ error: Optional[ErrorInfo]++class ConfigManagementTools:"""配置管理工具类"""@@ -23,7 +38,7 @@ class ConfigManagementTools:"""self.data_service = DataService(project_root)- def get_current_config(self, section: Optional[str] = None) -> Dict:+ def get_current_config(self, section: Optional[str] = None) -> ConfigResult:"""获取当前系统配置@@ -45,22 +60,24 @@ class ConfigManagementTools:# 获取配置config = self.data_service.get_current_config(section=section)- return {- "config": config,- "section": section,- "success": True- }+ return ConfigResult(+ success=True,+ config=config,+ section=section,+ error=None+ )except MCPError as e:- return {- "success": False,- "error": e.to_dict()- }+ return ConfigResult(+ success=False,+ config=None,+ section=None,+ error=e.to_dict()+ )except Exception as e:- return {- "success": False,- "error": {- "code": "INTERNAL_ERROR",- "message": str(e)- }- }+ return ConfigResult(+ success=False,+ config=None,+ section=None,+ error={"code": "INTERNAL_ERROR", "message": str(e), "suggestion": "请查看服务日志获取详细信息"}+ )diff --git a/mcp_server/utils/errors.py b/mcp_server/utils/errors.pyindex fe80d7eb7e..c8c74e8a88 100644--- a/mcp_server/utils/errors.py+++ b/mcp_server/utils/errors.py@@ -4,7 +4,25 @@定义MCP Server使用的所有自定义异常类型。"""-from typing import Optional+from typing import Optional, List, Callable+++# ==================== 延迟加载支持的平台列表 ====================++_get_supported_platforms: Optional[Callable[[], List[str]]] = None+++def _load_supported_platforms() -> List[str]:+ """延迟加载支持的平台列表"""+ global _get_supported_platforms+ if _get_supported_platforms is None:+ try:+ from .validators import get_supported_platforms+ _get_supported_platforms = get_supported_platforms+ except ImportError:+ # 降级:返回空列表+ return []+ return _get_supported_platforms()class MCPError(Exception):@@ -64,10 +82,12 @@ class PlatformNotSupportedError(MCPError):"""平台不支持错误"""def __init__(self, platform: str):+ supported = _load_supported_platforms()+ suggestion = f"支持的平台: {', '.join(supported)}" if supported else "请检查 config/config.yaml 中的平台配置"super().__init__(message=f"平台 '{platform}' 不受支持",code="PLATFORM_NOT_SUPPORTED",- suggestion="支持的平台: zhihu, weibo, douyin, bilibili, baidu, toutiao, qq, 36kr, sspai, hellogithub, thepaper"+ suggestion=suggestion)diff --git a/pyproject.toml b/pyproject.tomlindex b994ed3f94..9a6228f09d 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "trendradar"-version = "5.0.0"+version = "5.2.0"description = "TrendRadar - 热点新闻聚合与分析工具"requires-python = ">=3.10"dependencies = [diff --git a/trendradar/__init__.py b/trendradar/__init__.pyindex db65053e95..10b28a3f40 100644--- a/trendradar/__init__.py+++ b/trendradar/__init__.py@@ -9,5 +9,5 @@ TrendRadar - 热点新闻聚合与分析工具from trendradar.context import AppContext-__version__ = "5.0.0"+__version__ = "5.2.0"__all__ = ["AppContext", "__version__"]diff --git a/trendradar/__main__.py b/trendradar/__main__.pyindex 36f68c1ddd..1809182cfa 100644--- a/trendradar/__main__.py+++ b/trendradar/__main__.py@@ -76,29 +76,20 @@ class NewsAnalyzer:"incremental": {"mode_name": "增量模式","description": "增量模式(只关注新增新闻,无新增时不推送)",- "realtime_report_type": "实时增量",- "summary_report_type": "当日汇总",- "should_send_realtime": True,- "should_generate_summary": True,- "summary_mode": "daily",+ "report_type": "增量分析",+ "should_send_notification": True,},"current": {"mode_name": "当前榜单模式","description": "当前榜单模式(当前榜单匹配新闻 + 新增新闻区域 + 按时推送)",- "realtime_report_type": "实时当前榜单",- "summary_report_type": "当前榜单汇总",- "should_send_realtime": True,- "should_generate_summary": True,- "summary_mode": "current",+ "report_type": "当前榜单",+ "should_send_notification": True,},"daily": {- "mode_name": "当日汇总模式",- "description": "当日汇总模式(所有匹配新闻 + 新增新闻区域 + 按时推送)",- "realtime_report_type": "",- "summary_report_type": "当日汇总",- "should_send_realtime": False,- "should_generate_summary": True,- "summary_mode": "daily",+ "mode_name": "全天汇总模式",+ "description": "全天汇总模式(所有匹配新闻 + 新增新闻区域 + 按时推送)",+ "report_type": "全天汇总",+ "should_send_notification": True,},}@@ -210,6 +201,7 @@ class NewsAnalyzer:(cfg["NTFY_SERVER_URL"] and cfg["NTFY_TOPIC"]),cfg["BARK_URL"],cfg["SLACK_WEBHOOK_URL"],+ cfg["GENERIC_WEBHOOK_URL"],])@@ -244,13 +236,15 @@ class NewsAnalyzer:id_to_name: Optional[Dict],) -> Optional[AIAnalysisResult]:"""执行 AI 分析"""- ai_config = self.ctx.config.get("AI_ANALYSIS", {})- if not ai_config.get("ENABLED", False):+ analysis_config = self.ctx.config.get("AI_ANALYSIS", {})+ if not analysis_config.get("ENABLED", False):return Noneprint("[AI] 正在进行 AI 分析...")try:- analyzer = AIAnalyzer(ai_config, self.ctx.get_time)+ ai_config = self.ctx.config.get("AI", {})+ debug_mode = self.ctx.config.get("DEBUG", False)+ analyzer = AIAnalyzer(ai_config, analysis_config, self.ctx.get_time, debug=debug_mode)# 提取平台列表platforms = list(id_to_name.values()) if id_to_name else []@@ -369,8 +363,11 @@ class NewsAnalyzer:Returns:独立展示数据字典,如果未启用返回 None"""- standalone_config = self.ctx.config.get("STANDALONE_DISPLAY", {})- if not standalone_config.get("ENABLED", False):+ display_config = self.ctx.config.get("DISPLAY", {})+ regions = display_config.get("REGIONS", {})+ standalone_config = display_config.get("STANDALONE", {})++ if not regions.get("STANDALONE", False):return Noneplatform_ids = standalone_config.get("PLATFORMS", [])@@ -504,13 +501,13 @@ class NewsAnalyzer:filter_words: List[str],id_to_name: Dict,failed_ids: Optional[List] = None,- is_daily_summary: bool = False,global_filters: Optional[List[str]] = None,quiet: bool = False,rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,- ) -> Tuple[List[Dict], Optional[str]]:- """统一的分析流水线:数据处理 → 统计计算 → HTML生成"""+ standalone_data: Optional[Dict] = None,+ ) -> Tuple[List[Dict], Optional[str], Optional[AIAnalysisResult]]:+ """统一的分析流水线:数据处理 → 统计计算 → AI分析 → HTML生成"""# 统计计算(使用 AppContext)stats, total_titles = self.ctx.count_frequency(@@ -533,6 +530,17 @@ class NewsAnalyzer:self.ctx.rank_threshold,)+ # AI 分析(如果启用,用于 HTML 报告)+ ai_result = None+ ai_config = self.ctx.config.get("AI_ANALYSIS", {})+ if ai_config.get("ENABLED", False) and stats:+ # 获取模式策略来确定报告类型+ mode_strategy = self._get_mode_strategy()+ report_type = mode_strategy["report_type"]+ ai_result = self._run_ai_analysis(+ stats, rss_items, mode, report_type, id_to_name+ )+# HTML生成(如果启用)html_file = Noneif self.ctx.config["STORAGE"]["FORMATS"]["HTML"]:@@ -543,13 +551,14 @@ class NewsAnalyzer:new_titles=new_titles,id_to_name=id_to_name,mode=mode,- is_daily_summary=is_daily_summary,update_info=self.update_info if self.ctx.config["SHOW_VERSION_UPDATE"] else None,rss_items=rss_items,rss_new_items=rss_new_items,+ ai_analysis=ai_result,+ standalone_data=standalone_data,)- return stats, html_file+ return stats, html_file, ai_resultdef _send_notification_if_needed(self,@@ -563,6 +572,7 @@ class NewsAnalyzer:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,standalone_data: Optional[Dict] = None,+ ai_result: Optional[AIAnalysisResult] = None,) -> bool:"""统一的通知发送逻辑,包含所有判断条件,支持热榜+RSS合并推送+AI分析+独立展示区"""has_notification = self._has_notification_configured()@@ -612,13 +622,13 @@ class NewsAnalyzer:else:print(f"推送窗口控制:今天首次推送")- # AI 分析(如果启用)- ai_result = None- ai_config = cfg.get("AI_ANALYSIS", {})- if ai_config.get("ENABLED", False):- ai_result = self._run_ai_analysis(- stats, rss_items, mode, report_type, id_to_name- )+ # AI 分析:优先使用传入的结果,避免重复分析+ if ai_result is None:+ ai_config = cfg.get("AI_ANALYSIS", {})+ if ai_config.get("ENABLED", False):+ ai_result = self._run_ai_analysis(+ stats, rss_items, mode, report_type, id_to_name+ )# 准备报告数据report_data = self.ctx.prepare_report(stats, failed_ids, new_titles, id_to_name, mode)@@ -666,125 +676,21 @@ class NewsAnalyzer:and not has_any_content):mode_strategy = self._get_mode_strategy()- if "实时" in report_type:- if self.report_mode == "incremental":- has_new = bool(- new_titles and any(len(titles) > 0 for titles in new_titles.values())- )- if not has_new and not has_rss_content:- print("跳过实时推送通知:增量模式下未检测到新增的新闻和RSS")- elif not has_new:- print("跳过实时推送通知:增量模式下新增新闻未匹配到关键词")- else:- print(- f"跳过实时推送通知:{mode_strategy['mode_name']}下未检测到匹配的新闻"- )+ if self.report_mode == "incremental":+ has_new = bool(+ new_titles and any(len(titles) > 0 for titles in new_titles.values())+ )+ if not has_new and not has_rss_content:+ print("跳过通知:增量模式下未检测到新增的新闻和RSS")+ elif not has_new:+ print("跳过通知:增量模式下新增新闻未匹配到关键词")else:print(- f"跳过{mode_strategy['summary_report_type']}通知:未匹配到有效的新闻内容"+ f"跳过通知:{mode_strategy['mode_name']}下未检测到匹配的新闻")return False- def _generate_summary_report(- self,- mode_strategy: Dict,- rss_items: Optional[List[Dict]] = None,- rss_new_items: Optional[List[Dict]] = None,- ) -> Optional[str]:- """生成汇总报告(带通知,支持RSS合并)"""- summary_type = (- "当前榜单汇总" if mode_strategy["summary_mode"] == "current" else "当日汇总"- )- print(f"生成{summary_type}报告...")-- # 加载分析数据- analysis_data = self._load_analysis_data()- if not analysis_data:- return None-- all_results, id_to_name, title_info, new_titles, word_groups, filter_words, global_filters = (- analysis_data- )-- # 运行分析流水线- stats, html_file = self._run_analysis_pipeline(- all_results,- mode_strategy["summary_mode"],- title_info,- new_titles,- word_groups,- filter_words,- id_to_name,- is_daily_summary=True,- global_filters=global_filters,- rss_items=rss_items,- rss_new_items=rss_new_items,- )-- if html_file:- print(f"{summary_type}报告已生成: {html_file}")-- # 准备独立展示区数据- standalone_data = self._prepare_standalone_data(- all_results, id_to_name, title_info, rss_items- )-- # 发送通知(合并RSS+独立展示区)- self._send_notification_if_needed(- stats,- mode_strategy["summary_report_type"],- mode_strategy["summary_mode"],- failed_ids=[],- new_titles=new_titles,- id_to_name=id_to_name,- html_file_path=html_file,- rss_items=rss_items,- rss_new_items=rss_new_items,- standalone_data=standalone_data,- )-- return html_file-- def _generate_summary_html(- self,- mode: str = "daily",- rss_items: Optional[List[Dict]] = None,- rss_new_items: Optional[List[Dict]] = None,- ) -> Optional[str]:- """生成汇总HTML"""- summary_type = "当前榜单汇总" if mode == "current" else "当日汇总"- print(f"生成{summary_type}HTML...")-- # 加载分析数据(静默模式,避免重复输出日志)- analysis_data = self._load_analysis_data(quiet=True)- if not analysis_data:- return None-- all_results, id_to_name, title_info, new_titles, word_groups, filter_words, global_filters = (- analysis_data- )-- # 运行分析流水线(静默模式,避免重复输出日志)- _, html_file = self._run_analysis_pipeline(- all_results,- mode,- title_info,- new_titles,- word_groups,- filter_words,- id_to_name,- is_daily_summary=True,- global_filters=global_filters,- quiet=True,- rss_items=rss_items,- rss_new_items=rss_new_items,- )-- if html_file:- print(f"{summary_type}HTML已生成: {html_file}")- return html_file-def _initialize_and_check_config(self) -> None:"""通用初始化和配置检查"""now = self.ctx.get_time()@@ -848,23 +754,24 @@ class NewsAnalyzer:return results, id_to_name, failed_ids- def _crawl_rss_data(self) -> Tuple[Optional[List[Dict]], Optional[List[Dict]]]:+ def _crawl_rss_data(self) -> Tuple[Optional[List[Dict]], Optional[List[Dict]], Optional[List[Dict]]]:"""执行 RSS 数据抓取Returns:- (rss_items, rss_new_items) 元组:+ (rss_items, rss_new_items, raw_rss_items) 元组:- rss_items: 统计条目列表(按模式处理,用于统计区块)- rss_new_items: 新增条目列表(用于新增区块)- 如果未启用或失败返回 (None, None)+ - raw_rss_items: 原始 RSS 条目列表(用于独立展示区)+ 如果未启用或失败返回 (None, None, None)"""if not self.ctx.rss_enabled:- return None, None+ return None, None, Nonerss_feeds = self.ctx.rss_feedsif not rss_feeds:print("[RSS] 未配置任何 RSS 源")- return None, None+ return None, None, Nonetry:from trendradar.crawler.rss import RSSFetcher, RSSFeedConfig@@ -900,7 +807,7 @@ class NewsAnalyzer:if not feeds:print("[RSS] 没有启用的 RSS 源")- return None, None+ return None, None, None# 创建抓取器rss_config = self.ctx.rss_config@@ -935,17 +842,17 @@ class NewsAnalyzer:return self._process_rss_data_by_mode(rss_data)else:print(f"[RSS] 数据保存失败")- return None, None+ return None, None, Noneexcept ImportError as e:print(f"[RSS] 缺少依赖: {e}")print("[RSS] 请安装 feedparser: pip install feedparser")- return None, None+ return None, None, Noneexcept Exception as e:print(f"[RSS] 抓取失败: {e}")- return None, None+ return None, None, None- def _process_rss_data_by_mode(self, rss_data) -> Tuple[Optional[List[Dict]], Optional[List[Dict]]]:+ def _process_rss_data_by_mode(self, rss_data) -> Tuple[Optional[List[Dict]], Optional[List[Dict]], Optional[List[Dict]]]:"""按报告模式处理 RSS 数据,返回与热榜相同格式的统计结构@@ -958,17 +865,15 @@ class NewsAnalyzer:rss_data: 当前抓取的 RSSData 对象Returns:- (rss_stats, rss_new_stats) 元组:+ (rss_stats, rss_new_stats, raw_rss_items) 元组:- rss_stats: RSS 关键词统计列表(与热榜 stats 格式一致)- rss_new_stats: RSS 新增关键词统计列表(与热榜 stats 格式一致)+ - raw_rss_items: 原始 RSS 条目列表(用于独立展示区)"""from trendradar.core.analyzer import count_rss_frequency- rss_config = self.ctx.rss_config-- # 检查是否启用 RSS 通知- if not rss_config.get("NOTIFICATION", {}).get("ENABLED", False):- return None, None+ # 从 display.regions.rss 统一控制 RSS 分析和展示+ rss_display_enabled = self.ctx.config.get("DISPLAY", {}).get("REGIONS", {}).get("RSS", True)# 加载关键词配置try:@@ -982,8 +887,28 @@ class NewsAnalyzer:rss_stats = Nonerss_new_stats = None+ raw_rss_items = None # 原始 RSS 条目列表(用于独立展示区)++ # 1. 首先获取原始条目(用于独立展示区,不受 display.regions.rss 影响)+ # 根据模式获取原始条目+ if self.report_mode == "incremental":+ new_items_dict = self.storage_manager.detect_new_rss_items(rss_data)+ if new_items_dict:+ raw_rss_items = self._convert_rss_items_to_list(new_items_dict, rss_data.id_to_name)+ elif self.report_mode == "current":+ latest_data = self.storage_manager.get_latest_rss_data(rss_data.date)+ if latest_data:+ raw_rss_items = self._convert_rss_items_to_list(latest_data.items, latest_data.id_to_name)+ else: # daily+ all_data = self.storage_manager.get_rss_data(rss_data.date)+ if all_data:+ raw_rss_items = self._convert_rss_items_to_list(all_data.items, all_data.id_to_name)- # 1. 首先获取新增条目(所有模式都需要)+ # 如果 RSS 展示未启用,跳过关键词分析,只返回原始条目用于独立展示区+ if not rss_display_enabled:+ return None, None, raw_rss_items++ # 2. 获取新增条目(用于统计)new_items_dict = self.storage_manager.detect_new_rss_items(rss_data)new_items_list = Noneif new_items_dict:@@ -991,12 +916,12 @@ class NewsAnalyzer:if new_items_list:print(f"[RSS] 检测到 {len(new_items_list)} 条新增")- # 2. 根据模式获取统计条目+ # 3. 根据模式获取统计条目if self.report_mode == "incremental":# 增量模式:统计条目就是新增条目if not new_items_list:print("[RSS] 增量模式:没有新增 RSS 条目")- return None, None+ return None, None, raw_rss_itemsrss_stats, total = count_rss_frequency(rss_items=new_items_list,@@ -1012,18 +937,18 @@ class NewsAnalyzer:)if not rss_stats:print("[RSS] 增量模式:关键词匹配后没有内容")- return None, None+ # 即使关键词匹配为空,也返回原始条目用于独立展示区+ return None, None, raw_rss_itemselif self.report_mode == "current":# 当前榜单模式:统计=当前榜单所有条目- latest_data = self.storage_manager.get_latest_rss_data(rss_data.date)- if not latest_data:+ # raw_rss_items 已在前面获取+ if not raw_rss_items:print("[RSS] 当前榜单模式:没有 RSS 数据")- return None, None+ return None, None, None- all_items_list = self._convert_rss_items_to_list(latest_data.items, latest_data.id_to_name)rss_stats, total = count_rss_frequency(- rss_items=all_items_list,+ rss_items=raw_rss_items,word_groups=word_groups,filter_words=filter_words,global_filters=global_filters,@@ -1036,7 +961,8 @@ class NewsAnalyzer:)if not rss_stats:print("[RSS] 当前榜单模式:关键词匹配后没有内容")- return None, None+ # 即使关键词匹配为空,也返回原始条目用于独立展示区+ return None, None, raw_rss_items# 生成新增统计if new_items_list:@@ -1055,14 +981,13 @@ class NewsAnalyzer:else:# daily 模式:统计=当天所有条目- all_data = self.storage_manager.get_rss_data(rss_data.date)- if not all_data:+ # raw_rss_items 已在前面获取+ if not raw_rss_items:print("[RSS] 当日汇总模式:没有 RSS 数据")- return None, None+ return None, None, None- all_items_list = self._convert_rss_items_to_list(all_data.items, all_data.id_to_name)rss_stats, total = count_rss_frequency(- rss_items=all_items_list,+ rss_items=raw_rss_items,word_groups=word_groups,filter_words=filter_words,global_filters=global_filters,@@ -1075,7 +1000,8 @@ class NewsAnalyzer:)if not rss_stats:print("[RSS] 当日汇总模式:关键词匹配后没有内容")- return None, None+ # 即使关键词匹配为空,也返回原始条目用于独立展示区+ return None, None, raw_rss_items# 生成新增统计if new_items_list:@@ -1092,7 +1018,7 @@ class NewsAnalyzer:quiet=True,)- return rss_stats, rss_new_stats+ return rss_stats, rss_new_stats, raw_rss_itemsdef _convert_rss_items_to_list(self, items_dict: Dict, id_to_name: Dict) -> List[Dict]:"""将 RSS 条目字典转换为列表格式,并应用新鲜度过滤(用于推送)"""@@ -1170,12 +1096,6 @@ class NewsAnalyzer:passreturn rss_items- def _process_rss_report_and_notification(self, rss_data) -> None:- """处理 RSS 报告生成和通知发送(独立推送,已废弃)"""- # 此方法保留用于向后兼容,但不再使用- # RSS 现在与热榜合并推送- pass-def _generate_rss_html_report(self, rss_items: list, feeds_info: dict) -> str:"""生成 RSS HTML 报告"""try:@@ -1189,10 +1109,10 @@ class NewsAnalyzer:get_time_func=self.ctx.get_time,)- # 保存 HTML 文件+ # 保存 HTML 文件(扁平化结构:output/html/日期/)date_folder = self.ctx.format_date()time_filename = self.ctx.format_time()- output_dir = Path("output") / date_folder / "html"+ output_dir = Path("output") / "html" / date_folderoutput_dir.mkdir(parents=True, exist_ok=True)file_path = output_dir / f"rss_{time_filename}.html"@@ -1210,8 +1130,14 @@ class NewsAnalyzer:self, mode_strategy: Dict, results: Dict, id_to_name: Dict, failed_ids: List,rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,+ raw_rss_items: Optional[List[Dict]] = None,) -> Optional[str]:- """执行模式特定逻辑,支持热榜+RSS合并推送"""+ """执行模式特定逻辑,支持热榜+RSS合并推送++ 简化后的逻辑:+ - 每次运行都生成 HTML 报告(时间戳快照 + latest/{mode}.html + index.html)+ - 根据模式发送通知+ """# 获取当前监控平台ID列表current_platform_ids = self.ctx.platform_ids@@ -1221,9 +1147,13 @@ class NewsAnalyzer:self.ctx.save_titles(results, id_to_name, failed_ids)word_groups, filter_words, global_filters = self.ctx.load_frequency_words()- # current模式下,实时推送需要使用完整的历史数据来保证统计信息的完整性+ html_file = None+ stats = []+ ai_result = None+ title_info = None++ # current 模式需要使用完整的历史数据if self.report_mode == "current":- # 加载完整的历史数据(已按当前平台过滤)analysis_data = self._load_analysis_data()if analysis_data:(@@ -1240,7 +1170,12 @@ class NewsAnalyzer:f"current模式:使用过滤后的历史数据,包含平台:{list(all_results.keys())}")- stats, html_file = self._run_analysis_pipeline(+ # 使用历史数据准备独立展示区数据(包含完整的 title_info)+ standalone_data = self._prepare_standalone_data(+ all_results, historical_id_to_name, historical_title_info, raw_rss_items+ )++ stats, html_file, ai_result = self._run_analysis_pipeline(all_results,self.report_mode,historical_title_info,@@ -1252,38 +1187,83 @@ class NewsAnalyzer:global_filters=global_filters,rss_items=rss_items,rss_new_items=rss_new_items,+ standalone_data=standalone_data,)combined_id_to_name = {**historical_id_to_name, **id_to_name}-- if html_file:- print(f"HTML报告已生成: {html_file}")-- # 发送实时通知(使用完整历史数据的统计结果,合并RSS+独立展示区)- summary_html = None- if mode_strategy["should_send_realtime"]:- # 准备独立展示区数据- standalone_data = self._prepare_standalone_data(- all_results, combined_id_to_name, historical_title_info, rss_items- )- self._send_notification_if_needed(- stats,- mode_strategy["realtime_report_type"],- self.report_mode,- failed_ids=failed_ids,- new_titles=historical_new_titles,- id_to_name=combined_id_to_name,- html_file_path=html_file,- rss_items=rss_items,- rss_new_items=rss_new_items,- standalone_data=standalone_data,- )+ new_titles = historical_new_titles+ id_to_name = combined_id_to_name+ title_info = historical_title_info+ results = all_resultselse:print("❌ 严重错误:无法读取刚保存的数据文件")raise RuntimeError("数据一致性检查失败:保存后立即读取失败")+ elif self.report_mode == "daily":+ # daily 模式:使用全天累计数据+ analysis_data = self._load_analysis_data()+ if analysis_data:+ (+ all_results,+ historical_id_to_name,+ historical_title_info,+ historical_new_titles,+ _,+ _,+ _,+ ) = analysis_data++ # 使用历史数据准备独立展示区数据(包含完整的 title_info)+ standalone_data = self._prepare_standalone_data(+ all_results, historical_id_to_name, historical_title_info, raw_rss_items+ )++ stats, html_file, ai_result = self._run_analysis_pipeline(+ all_results,+ self.report_mode,+ historical_title_info,+ historical_new_titles,+ word_groups,+ filter_words,+ historical_id_to_name,+ failed_ids=failed_ids,+ global_filters=global_filters,+ rss_items=rss_items,+ rss_new_items=rss_new_items,+ standalone_data=standalone_data,+ )++ combined_id_to_name = {**historical_id_to_name, **id_to_name}+ new_titles = historical_new_titles+ id_to_name = combined_id_to_name+ title_info = historical_title_info+ results = all_results+ else:+ # 没有历史数据时使用当前数据+ title_info = self._prepare_current_title_info(results, time_info)+ standalone_data = self._prepare_standalone_data(+ results, id_to_name, title_info, raw_rss_items+ )+ stats, html_file, ai_result = self._run_analysis_pipeline(+ results,+ self.report_mode,+ title_info,+ new_titles,+ word_groups,+ filter_words,+ id_to_name,+ failed_ids=failed_ids,+ global_filters=global_filters,+ rss_items=rss_items,+ rss_new_items=rss_new_items,+ standalone_data=standalone_data,+ )else:+ # incremental 模式:只使用当前抓取的数据title_info = self._prepare_current_title_info(results, time_info)- stats, html_file = self._run_analysis_pipeline(+ standalone_data = self._prepare_standalone_data(+ results, id_to_name, title_info, raw_rss_items+ )+ stats, html_file, ai_result = self._run_analysis_pipeline(results,self.report_mode,title_info,@@ -1295,63 +1275,41 @@ class NewsAnalyzer:global_filters=global_filters,rss_items=rss_items,rss_new_items=rss_new_items,+ standalone_data=standalone_data,)- if html_file:- print(f"HTML报告已生成: {html_file}")- # 发送实时通知(如果需要,合并RSS+独立展示区)- summary_html = None- if mode_strategy["should_send_realtime"]:- # 准备独立展示区数据- standalone_data = self._prepare_standalone_data(- results, id_to_name, title_info, rss_items- )- self._send_notification_if_needed(- stats,- mode_strategy["realtime_report_type"],- self.report_mode,- failed_ids=failed_ids,- new_titles=new_titles,- id_to_name=id_to_name,- html_file_path=html_file,- rss_items=rss_items,- rss_new_items=rss_new_items,- standalone_data=standalone_data,- )+ if html_file:+ print(f"HTML报告已生成: {html_file}")+ print(f"最新报告已更新: output/html/latest/{self.report_mode}.html")- # 生成汇总报告(如果需要)- summary_html = None- if mode_strategy["should_generate_summary"]:- if mode_strategy["should_send_realtime"]:- # 如果已经发送了实时通知,汇总只生成HTML不发送通知- summary_html = self._generate_summary_html(- mode_strategy["summary_mode"],- rss_items=rss_items,- rss_new_items=rss_new_items,- )- else:- # daily模式:直接生成汇总报告并发送通知(合并RSS)- summary_html = self._generate_summary_report(- mode_strategy, rss_items=rss_items, rss_new_items=rss_new_items- )+ # 发送通知+ if mode_strategy["should_send_notification"]:+ standalone_data = self._prepare_standalone_data(+ results, id_to_name, title_info, raw_rss_items+ )+ self._send_notification_if_needed(+ stats,+ mode_strategy["report_type"],+ self.report_mode,+ failed_ids=failed_ids,+ new_titles=new_titles,+ id_to_name=id_to_name,+ html_file_path=html_file,+ rss_items=rss_items,+ rss_new_items=rss_new_items,+ standalone_data=standalone_data,+ ai_result=ai_result,+ )# 打开浏览器(仅在非容器环境)if self._should_open_browser() and html_file:- if summary_html:- summary_url = "file://" + str(Path(summary_html).resolve())- print(f"正在打开汇总报告: {summary_url}")- webbrowser.open(summary_url)- else:- file_url = "file://" + str(Path(html_file).resolve())- print(f"正在打开HTML报告: {file_url}")- webbrowser.open(file_url)+ file_url = "file://" + str(Path(html_file).resolve())+ print(f"正在打开HTML报告: {file_url}")+ webbrowser.open(file_url)elif self.is_docker_container and html_file:- if summary_html:- print(f"汇总报告已生成(Docker环境): {summary_html}")- else:- print(f"HTML报告已生成(Docker环境): {html_file}")+ print(f"HTML报告已生成(Docker环境): {html_file}")- return summary_html+ return html_filedef run(self) -> None:"""执行分析流程"""@@ -1363,13 +1321,14 @@ class NewsAnalyzer:# 抓取热榜数据results, id_to_name, failed_ids = self._crawl_data()- # 抓取 RSS 数据(如果启用),返回统计条目和新增条目用于合并推送- rss_items, rss_new_items = self._crawl_rss_data()+ # 抓取 RSS 数据(如果启用),返回统计条目、新增条目和原始条目+ rss_items, rss_new_items, raw_rss_items = self._crawl_rss_data()# 执行模式策略,传递 RSS 数据用于合并推送self._execute_mode_strategy(mode_strategy, results, id_to_name, failed_ids,- rss_items=rss_items, rss_new_items=rss_new_items+ rss_items=rss_items, rss_new_items=rss_new_items,+ raw_rss_items=raw_rss_items)except Exception as e:diff --git a/trendradar/ai/__init__.py b/trendradar/ai/__init__.pyindex 69c3f97371..a4f67018c9 100644--- a/trendradar/ai/__init__.py+++ b/trendradar/ai/__init__.py@@ -1,27 +1,36 @@# coding=utf-8"""-TrendRadar AI 分析模块+TrendRadar AI 模块-提供 AI 大模型对热点新闻的深度分析功能+提供 AI 大模型对热点新闻的深度分析和翻译功能"""from .analyzer import AIAnalyzer, AIAnalysisResult+from .translator import AITranslator, TranslationResult, BatchTranslationResultfrom .formatter import (get_ai_analysis_renderer,render_ai_analysis_markdown,render_ai_analysis_feishu,render_ai_analysis_dingtalk,render_ai_analysis_html,+ render_ai_analysis_html_rich,render_ai_analysis_plain,)__all__ = [+ # 分析器"AIAnalyzer","AIAnalysisResult",+ # 翻译器+ "AITranslator",+ "TranslationResult",+ "BatchTranslationResult",+ # 格式化"get_ai_analysis_renderer","render_ai_analysis_markdown","render_ai_analysis_feishu","render_ai_analysis_dingtalk","render_ai_analysis_html",+ "render_ai_analysis_html_rich","render_ai_analysis_plain",]diff --git a/trendradar/ai/analyzer.py b/trendradar/ai/analyzer.pyindex 29686cebb4..c4d62678f3 100644--- a/trendradar/ai/analyzer.py+++ b/trendradar/ai/analyzer.py@@ -16,16 +16,18 @@ from typing import Any, Callable, Dict, List, Optional@dataclassclass AIAnalysisResult:"""AI 分析结果"""- summary: str = "" # 热点趋势概述- keyword_analysis: str = "" # 关键词热度分析- sentiment: str = "" # 情感倾向分析- cross_platform: str = "" # 跨平台关联- impact: str = "" # 潜在影响评估- signals: str = "" # 值得关注的信号- conclusion: str = "" # 总结与建议+ # 新版 5 核心板块+ core_trends: str = "" # 核心热点与舆情态势+ sentiment_controversy: str = "" # 舆论风向与争议+ signals: str = "" # 异动与弱信号+ rss_insights: str = "" # RSS 深度洞察+ outlook_strategy: str = "" # 研判与策略建议++ # 基础元数据raw_response: str = "" # 原始响应success: bool = False # 是否成功error: str = "" # 错误信息+# 新闻数量统计total_news: int = 0 # 总新闻数(热榜+RSS)analyzed_news: int = 0 # 实际分析的新闻数@@ -37,30 +39,57 @@ class AIAnalysisResult:class AIAnalyzer:"""AI 分析器"""- def __init__(self, config: Dict[str, Any], get_time_func: Callable):+ def __init__(+ self,+ ai_config: Dict[str, Any],+ analysis_config: Dict[str, Any],+ get_time_func: Callable,+ debug: bool = False,+ ):"""初始化 AI 分析器Args:- config: AI 分析配置+ ai_config: AI 模型共享配置(provider, api_key, model 等)+ analysis_config: AI 分析功能配置(language, prompt_file 等)get_time_func: 获取当前时间的函数+ debug: 是否开启调试模式"""- self.config = config+ self.ai_config = ai_config+ self.analysis_config = analysis_configself.get_time_func = get_time_func-- # 从配置或环境变量获取 API Key- self.api_key = config.get("API_KEY") or os.environ.get("AI_API_KEY", "")- self.provider = config.get("PROVIDER", "openai")- self.model = config.get("MODEL", "gpt-4o-mini")- self.base_url = config.get("BASE_URL", "")- self.timeout = config.get("TIMEOUT", 90)- self.max_news = config.get("MAX_NEWS_FOR_ANALYSIS", 50)- self.include_rss = config.get("INCLUDE_RSS", True)- self.push_mode = config.get("PUSH_MODE", "both")+ self.debug = debug++ # 从共享配置获取模型参数+ self.api_key = ai_config.get("API_KEY") or os.environ.get("AI_API_KEY", "")+ self.provider = ai_config.get("PROVIDER", "deepseek")+ self.model = ai_config.get("MODEL", "deepseek-chat")+ self.base_url = ai_config.get("BASE_URL", "")+ self.timeout = ai_config.get("TIMEOUT", 90)+ self.temperature = ai_config.get("TEMPERATURE", 1.0)+ self.max_tokens = ai_config.get("MAX_TOKENS", 5000)++ # 从分析配置获取功能参数+ self.max_news = analysis_config.get("MAX_NEWS_FOR_ANALYSIS", 50)+ self.include_rss = analysis_config.get("INCLUDE_RSS", True)+ self.include_rank_timeline = analysis_config.get("INCLUDE_RANK_TIMELINE", False)+ self.language = analysis_config.get("LANGUAGE", "Chinese")++ # 额外的自定义参数(支持字典或 JSON 字符串)+ self.extra_params = ai_config.get("EXTRA_PARAMS", {})+ if isinstance(self.extra_params, str) and self.extra_params.strip():+ try:+ self.extra_params = json.loads(self.extra_params)+ except json.JSONDecodeError:+ print(f"[AI] 解析 extra_params 失败,将忽略: {self.extra_params}")+ self.extra_params = {}++ if not isinstance(self.extra_params, dict):+ self.extra_params = {}# 加载提示词模板self.system_prompt, self.user_prompt_template = self._load_prompt_template(- config.get("PROMPT_FILE", "ai_analysis_prompt.txt")+ analysis_config.get("PROMPT_FILE", "ai_analysis_prompt.txt"))def _load_prompt_template(self, prompt_file: str) -> tuple:@@ -124,10 +153,10 @@ class AIAnalyzer:)# 准备新闻内容并获取统计数据- news_content, hotlist_total, rss_total, analyzed_count = self._prepare_news_content(stats, rss_stats)+ news_content, rss_content, hotlist_total, rss_total, analyzed_count = self._prepare_news_content(stats, rss_stats)total_news = hotlist_total + rss_total- if not news_content:+ if not news_content and not rss_content:return AIAnalysisResult(success=False,error="没有可分析的新闻内容",@@ -155,11 +184,29 @@ class AIAnalyzer:user_prompt = user_prompt.replace("{platforms}", ", ".join(platforms) if platforms else "多平台")user_prompt = user_prompt.replace("{keywords}", ", ".join(keywords[:20]) if keywords else "无")user_prompt = user_prompt.replace("{news_content}", news_content)+ user_prompt = user_prompt.replace("{rss_content}", rss_content)+ user_prompt = user_prompt.replace("{language}", self.language)++ if self.debug:+ print("\n" + "=" * 80)+ print("[AI 调试] 发送给 AI 的完整提示词")+ print("=" * 80)+ if self.system_prompt:+ print("\n--- System Prompt ---")+ print(self.system_prompt)+ print("\n--- User Prompt ---")+ print(user_prompt)+ print("=" * 80 + "\n")# 调用 AI APItry:response = self._call_ai_api(user_prompt)result = self._parse_response(response)++ # 如果配置未启用 RSS 分析,强制清空 AI 返回的 RSS 洞察+ if not self.include_rss:+ result.rss_insights = ""+# 填充统计数据result.total_news = total_newsresult.hotlist_count = hotlist_total@@ -210,10 +257,12 @@ class AIAnalyzer:RSS 包含:来源、标题、发布时间Returns:- tuple: (content_str, hotlist_total, rss_total, analyzed_count)+ tuple: (news_content, rss_content, hotlist_total, rss_total, analyzed_count)"""- lines = []- count = 0+ news_lines = []+ rss_lines = []+ news_count = 0+ rss_count = 0# 计算总新闻数hotlist_total = sum(len(s.get("titles", [])) for s in stats) if stats else 0@@ -221,13 +270,11 @@ class AIAnalyzer:# 热榜内容if stats:- lines.append("### 热榜新闻")- lines.append("格式: [来源] 标题 | 排名:最高-最低 | 时间:首次~末次 | 出现:N次")for stat in stats:word = stat.get("word", "")titles = stat.get("titles", [])if word and titles:- lines.append(f"\n**{word}** ({len(titles)}条)")+ news_lines.append(f"\n**{word}** ({len(titles)}条)")for t in titles:if not isinstance(t, dict):continue@@ -238,7 +285,13 @@ class AIAnalyzer:# 来源source = t.get("source_name", t.get("source", ""))- # 排名范围+ # 构建行+ if source:+ line = f"- [{source}] {title}"+ else:+ line = f"- {title}"++ # 始终显示简化格式:排名范围 + 时间范围 + 出现次数ranks = t.get("ranks", [])if ranks:min_rank = min(ranks)@@ -247,37 +300,38 @@ class AIAnalyzer:else:rank_str = "-"- # 时间范围(简化显示)first_time = t.get("first_time", "")last_time = t.get("last_time", "")time_str = self._format_time_range(first_time, last_time)- # 出现次数appear_count = t.get("count", 1)- # 构建行:[来源] 标题 | 排名:X-Y | 时间:首次~末次 | 出现:N次- if source:- line = f"- [{source}] {title}"- else:- line = f"- {title}"line += f" | 排名:{rank_str} | 时间:{time_str} | 出现:{appear_count}次"- lines.append(line)- count += 1- if count >= self.max_news:+ # 开启完整时间线时,额外添加轨迹+ if self.include_rank_timeline:+ rank_timeline = t.get("rank_timeline", [])+ timeline_str = self._format_rank_timeline(rank_timeline)+ line += f" | 轨迹:{timeline_str}"++ news_lines.append(line)++ news_count += 1+ if news_count >= self.max_news:break- if count >= self.max_news:+ if news_count >= self.max_news:break- # RSS 内容(仅在启用时提交)- if self.include_rss and rss_stats and count < self.max_news:- lines.append("\n### RSS 订阅")- lines.append("格式: [来源] 标题 | 发布时间")+ # RSS 内容(仅在启用时构建)+ if self.include_rss and rss_stats:+ remaining = self.max_news - news_countfor stat in rss_stats:+ if rss_count >= remaining:+ breakword = stat.get("word", "")titles = stat.get("titles", [])if word and titles:- lines.append(f"\n**{word}** ({len(titles)}条)")+ rss_lines.append(f"\n**{word}** ({len(titles)}条)")for t in titles:if not isinstance(t, dict):continue@@ -298,15 +352,17 @@ class AIAnalyzer:line = f"- {title}"if time_display:line += f" | {time_display}"- lines.append(line)+ rss_lines.append(line)- count += 1- if count >= self.max_news:+ rss_count += 1+ if rss_count >= remaining:break- if count >= self.max_news:- break- return "\n".join(lines), hotlist_total, rss_total, count+ news_content = "\n".join(news_lines) if news_lines else ""+ rss_content = "\n".join(rss_lines) if rss_lines else ""+ total_count = news_count + rss_count++ return news_content, rss_content, hotlist_total, rss_total, total_countdef _format_time_range(self, first_time: str, last_time: str) -> str:"""格式化时间范围(简化显示,只保留时分)"""@@ -314,7 +370,6 @@ class AIAnalyzer:if not time_str:return "-"# 尝试提取 HH:MM 部分- # 格式可能是 "2026-01-04 12:30:00" 或 "12:30" 等if " " in time_str:parts = time_str.split(" ")if len(parts) >= 2:@@ -323,7 +378,11 @@ class AIAnalyzer:return time_part[:5] # HH:MMelif ":" in time_str:return time_str[:5]- return time_str[:5] if len(time_str) >= 5 else time_str+ # 处理 HH-MM 格式+ result = time_str[:5] if len(time_str) >= 5 else time_str+ if len(result) == 5 and result[2] == '-':+ result = result.replace('-', ':')+ return resultfirst = extract_time(first_time)last = extract_time(last_time)@@ -332,6 +391,24 @@ class AIAnalyzer:return firstreturn f"{first}~{last}"+ def _format_rank_timeline(self, rank_timeline: List[Dict]) -> str:+ """格式化排名时间线"""+ if not rank_timeline:+ return "-"++ parts = []+ for item in rank_timeline:+ time_str = item.get("time", "")+ if len(time_str) == 5 and time_str[2] == '-':+ time_str = time_str.replace('-', ':')+ rank = item.get("rank")+ if rank is None:+ parts.append(f"0({time_str})")+ else:+ parts.append(f"{rank}({time_str})")++ return "→".join(parts)+def _call_ai_api(self, user_prompt: str) -> str:"""调用 AI API"""if self.provider == "gemini":@@ -372,10 +449,16 @@ class AIAnalyzer:payload = {"model": self.model,"messages": messages,- "temperature": 0.7,- "max_tokens": 2000,+ "temperature": self.temperature,}+ # 某些 API 不支持 max_tokens+ if self.max_tokens:+ payload["max_tokens"] = self.max_tokens++ if self.extra_params:+ payload.update(self.extra_params)+response = requests.post(url,headers=headers,@@ -391,7 +474,6 @@ class AIAnalyzer:"""调用 Google Gemini API"""import requests- # Gemini API URL 格式: https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContentmodel = self.model or "gemini-1.5-flash"url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={self.api_key}"@@ -399,30 +481,33 @@ class AIAnalyzer:"Content-Type": "application/json",}- # 构建 Gemini 格式的消息- contents = []- if self.system_prompt:- contents.append({- "role": "user",- "parts": [{"text": f"System instruction: {self.system_prompt}"}]- })- contents.append({- "role": "model",- "parts": [{"text": "Understood. I will follow these instructions."}]- })- contents.append({- "role": "user",- "parts": [{"text": user_prompt}]- })-payload = {- "contents": contents,+ "contents": [{+ "role": "user",+ "parts": [{"text": user_prompt}]+ }],"generationConfig": {- "temperature": 0.7,- "maxOutputTokens": 2000,- }+ "temperature": self.temperature,+ },+ "safetySettings": [+ {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"},+ ]}+ if self.system_prompt:+ payload["system_instruction"] = {+ "parts": [{"text": self.system_prompt}]+ }++ if self.max_tokens:+ payload["generationConfig"]["maxOutputTokens"] = self.max_tokens++ if self.extra_params:+ payload["generationConfig"].update(self.extra_params)+response = requests.post(url,headers=headers,@@ -447,57 +532,50 @@ class AIAnalyzer:# 提取 JSON 部分json_str = response- # 尝试提取 ```json ... ``` 代码块if "```json" in response:parts = response.split("```json", 1)if len(parts) > 1:code_block = parts[1]- # 查找结束的 ```end_idx = code_block.find("```")if end_idx != -1:json_str = code_block[:end_idx]else:- json_str = code_block # 没有结束标记,使用剩余内容- # 尝试提取 ``` ... ``` 代码块+ json_str = code_blockelif "```" in response:- parts = response.split("```", 2) # 最多分割2次+ parts = response.split("```", 2)if len(parts) >= 2:json_str = parts[1]- # 清理 JSON 字符串json_str = json_str.strip()if not json_str:raise ValueError("提取的 JSON 内容为空")data = json.loads(json_str)- result.summary = data.get("summary", "")- result.keyword_analysis = data.get("keyword_analysis", "")- result.sentiment = data.get("sentiment", "")- result.cross_platform = data.get("cross_platform", "")- result.impact = data.get("impact", "")+ # 新版字段解析+ result.core_trends = data.get("core_trends", "")+ result.sentiment_controversy = data.get("sentiment_controversy", "")result.signals = data.get("signals", "")- result.conclusion = data.get("conclusion", "")+ result.rss_insights = data.get("rss_insights", "")+ result.outlook_strategy = data.get("outlook_strategy", "")+result.success = Trueexcept json.JSONDecodeError as e:- # JSON 解析失败,记录详细错误但仍使用原始文本error_context = json_str[max(0, e.pos - 30):e.pos + 30] if json_str and e.pos else ""result.error = f"JSON 解析错误 (位置 {e.pos}): {e.msg}"if error_context:result.error += f",上下文: ...{error_context}..."- # 使用原始响应作为 summary- result.summary = response[:1000] if len(response) > 1000 else response- result.success = True # 仍标记为成功,因为有内容可展示+ # 使用原始响应填充 core_trends,确保有输出+ result.core_trends = response[:500] + "..." if len(response) > 500 else response+ result.success = Trueexcept (IndexError, KeyError, TypeError, ValueError) as e:- # 其他解析错误result.error = f"响应解析错误: {type(e).__name__}: {str(e)}"- result.summary = response[:1000] if len(response) > 1000 else response+ result.core_trends = response[:500] if len(response) > 500 else responseresult.success = Trueexcept Exception as e:- # 未知错误result.error = f"解析时发生未知错误: {type(e).__name__}: {str(e)}"- result.summary = response[:1000] if len(response) > 1000 else response+ result.core_trends = response[:500] if len(response) > 500 else responseresult.success = Truereturn resultdiff --git a/trendradar/ai/formatter.py b/trendradar/ai/formatter.pyindex cf86c7eeb1..120df535dd 100644--- a/trendradar/ai/formatter.py+++ b/trendradar/ai/formatter.py@@ -6,6 +6,7 @@ AI 分析结果格式化模块"""import html as html_lib+import refrom .analyzer import AIAnalysisResult@@ -14,6 +15,46 @@ def _escape_html(text: str) -> str:return html_lib.escape(text) if text else ""+def _format_list_content(text: str) -> str:+ """+ 格式化列表内容,确保序号前有换行+ 例如将 "1. xxx 2. yyy" 转换为:+ 1. xxx+ 2. yyy+ """+ if not text:+ return ""++ # 去除首尾空白,防止 AI 返回的内容开头就有换行导致显示空行+ text = text.strip()++ # 1. 规范化:确保 "1." 后面有空格+ result = re.sub(r'(\d+)\.([^ \d])', r'\1. \2', text)++ # 2. 强制换行:匹配 "数字.",且前面不是换行符+ result = re.sub(r'(?<=[^\n])\s+(\d+\.)', r'\n\1', result)++ # 3. 处理 "1.**粗体**" 这种情况(虽然 Prompt 要求不输出 Markdown,但防御性处理)+ result = re.sub(r'(?<=[^\n])(\d+\.\*\*)', r'\n\1', result)++ # 4. 处理中文标点后的换行+ result = re.sub(r'([::;,。;,])\s*(\d+\.)', r'\1\n\2', result)++ # 5. 处理 "XX方面:"、"XX领域:" 等子标题换行+ # 只有在中文标点(句号、逗号、分号等)后才触发换行,避免破坏 "1. XX领域:" 格式+ result = re.sub(r'([。!?;,、])\s*([a-zA-Z0-9\u4e00-\u9fa5]+(方面|领域)[::])', r'\1\n\2', result)++ # 6. 处理 "【XX】:"(如【宏观主线】:) 前的换行,确保视觉分隔+ result = re.sub(r'(?<=[^\n])\s*(【[^】]+】[::])', r'\n\n\1', result)++ # 7. 在列表项之间增加视觉空行(将 \n数字. 替换为 \n\n数字.)+ # 但排除标题行(以冒号结尾)之后的情况,避免标题和第一项之间有空行+ # (?<![::]) 是负向后瞻,表示前面不能是冒号+ result = re.sub(r'(?<![::])\n(\d+\.)', r'\n\n\1', result)++ return result++def render_ai_analysis_markdown(result: AIAnalysisResult) -> str:"""渲染为通用 Markdown 格式(Telegram、企业微信、ntfy、Bark、Slack)"""if not result.success:@@ -21,26 +62,20 @@ def render_ai_analysis_markdown(result: AIAnalysisResult) -> str:lines = ["**✨ AI 热点分析**", ""]- if result.summary:- lines.extend(["**趋势概述**", result.summary, ""])-- if result.keyword_analysis:- lines.extend(["**热度走势**", result.keyword_analysis, ""])-- if result.sentiment:- lines.extend(["**情感倾向**", result.sentiment, ""])-- if result.cross_platform:- lines.extend(["**跨平台关联**", result.cross_platform, ""])+ if result.core_trends:+ lines.extend(["**核心热点态势**", _format_list_content(result.core_trends), ""])- if result.impact:- lines.extend(["**潜在影响**", result.impact, ""])+ if result.sentiment_controversy:+ lines.extend(["**舆论风向争议**", _format_list_content(result.sentiment_controversy), ""])if result.signals:- lines.extend(["**值得关注**", result.signals, ""])+ lines.extend(["**异动与弱信号**", _format_list_content(result.signals), ""])- if result.conclusion:- lines.extend(["**总结建议**", result.conclusion])+ if result.rss_insights:+ lines.extend(["**RSS 深度洞察**", _format_list_content(result.rss_insights), ""])++ if result.outlook_strategy:+ lines.extend(["**研判策略建议**", _format_list_content(result.outlook_strategy)])return "\n".join(lines)@@ -52,26 +87,20 @@ def render_ai_analysis_feishu(result: AIAnalysisResult) -> str:lines = ["**✨ AI 热点分析**", ""]- if result.summary:- lines.extend(["**趋势概述**", result.summary, ""])-- if result.keyword_analysis:- lines.extend(["**热度走势**", result.keyword_analysis, ""])-- if result.sentiment:- lines.extend(["**情感倾向**", result.sentiment, ""])+ if result.core_trends:+ lines.extend(["**核心热点态势**", _format_list_content(result.core_trends), ""])- if result.cross_platform:- lines.extend(["**跨平台关联**", result.cross_platform, ""])-- if result.impact:- lines.extend(["**潜在影响**", result.impact, ""])+ if result.sentiment_controversy:+ lines.extend(["**舆论风向争议**", _format_list_content(result.sentiment_controversy), ""])if result.signals:- lines.extend(["**值得关注**", result.signals, ""])+ lines.extend(["**异动与弱信号**", _format_list_content(result.signals), ""])++ if result.rss_insights:+ lines.extend(["**RSS 深度洞察**", _format_list_content(result.rss_insights), ""])- if result.conclusion:- lines.extend(["**总结建议**", result.conclusion])+ if result.outlook_strategy:+ lines.extend(["**研判策略建议**", _format_list_content(result.outlook_strategy)])return "\n".join(lines)@@ -83,26 +112,20 @@ def render_ai_analysis_dingtalk(result: AIAnalysisResult) -> str:lines = ["### ✨ AI 热点分析", ""]- if result.summary:- lines.extend(["#### 趋势概述", result.summary, ""])+ if result.core_trends:+ lines.extend(["#### 核心热点态势", _format_list_content(result.core_trends), ""])- if result.keyword_analysis:- lines.extend(["#### 热度走势", result.keyword_analysis, ""])-- if result.sentiment:- lines.extend(["#### 情感倾向", result.sentiment, ""])-- if result.cross_platform:- lines.extend(["#### 跨平台关联", result.cross_platform, ""])-- if result.impact:- lines.extend(["#### 潜在影响", result.impact, ""])+ if result.sentiment_controversy:+ lines.extend(["#### 舆论风向争议", _format_list_content(result.sentiment_controversy), ""])if result.signals:- lines.extend(["#### 值得关注", result.signals, ""])+ lines.extend(["#### 异动与弱信号", _format_list_content(result.signals), ""])- if result.conclusion:- lines.extend(["#### 总结建议", result.conclusion])+ if result.rss_insights:+ lines.extend(["#### RSS 深度洞察", _format_list_content(result.rss_insights), ""])++ if result.outlook_strategy:+ lines.extend(["#### 研判策略建议", _format_list_content(result.outlook_strategy)])return "\n".join(lines)@@ -114,59 +137,53 @@ def render_ai_analysis_html(result: AIAnalysisResult) -> str:html_parts = ['<div class="ai-analysis">', '<h3>✨ AI 热点分析</h3>']- if result.summary:- html_parts.extend([- '<div class="ai-section">',- '<h4>趋势概述</h4>',- f'<p>{_escape_html(result.summary)}</p>',- '</div>'- ])-- if result.keyword_analysis:+ if result.core_trends:+ content = _format_list_content(result.core_trends)+ content_html = _escape_html(content).replace("\n", "<br>")html_parts.extend(['<div class="ai-section">',- '<h4>热度走势</h4>',- f'<p>{_escape_html(result.keyword_analysis)}</p>',+ '<h4>核心热点态势</h4>',+ f'<div class="ai-content">{content_html}</div>','</div>'])- if result.sentiment:+ if result.sentiment_controversy:+ content = _format_list_content(result.sentiment_controversy)+ content_html = _escape_html(content).replace("\n", "<br>")html_parts.extend(['<div class="ai-section">',- '<h4>情感倾向</h4>',- f'<p>{_escape_html(result.sentiment)}</p>',+ '<h4>舆论风向争议</h4>',+ f'<div class="ai-content">{content_html}</div>','</div>'])- if result.cross_platform:- html_parts.extend([- '<div class="ai-section">',- '<h4>跨平台关联</h4>',- f'<p>{_escape_html(result.cross_platform)}</p>',- '</div>'- ])-- if result.impact:+ if result.signals:+ content = _format_list_content(result.signals)+ content_html = _escape_html(content).replace("\n", "<br>")html_parts.extend(['<div class="ai-section">',- '<h4>潜在影响</h4>',- f'<p>{_escape_html(result.impact)}</p>',+ '<h4>异动与弱信号</h4>',+ f'<div class="ai-content">{content_html}</div>','</div>'])- if result.signals:+ if result.rss_insights:+ content = _format_list_content(result.rss_insights)+ content_html = _escape_html(content).replace("\n", "<br>")html_parts.extend(['<div class="ai-section">',- '<h4>值得关注</h4>',- f'<p>{_escape_html(result.signals)}</p>',+ '<h4>RSS 深度洞察</h4>',+ f'<div class="ai-content">{content_html}</div>','</div>'])- if result.conclusion:+ if result.outlook_strategy:+ content = _format_list_content(result.outlook_strategy)+ content_html = _escape_html(content).replace("\n", "<br>")html_parts.extend(['<div class="ai-section ai-conclusion">',- '<h4>总结建议</h4>',- f'<p>{_escape_html(result.conclusion)}</p>',+ '<h4>研判策略建议</h4>',+ f'<div class="ai-content">{content_html}</div>','</div>'])@@ -179,28 +196,22 @@ def render_ai_analysis_plain(result: AIAnalysisResult) -> str:if not result.success:return f"AI 分析失败: {result.error}"- lines = ["【AI 热点分析】", ""]+ lines = ["【✨ AI 热点分析】", ""]- if result.summary:- lines.extend(["[趋势概述]", result.summary, ""])+ if result.core_trends:+ lines.extend(["[核心热点态势]", _format_list_content(result.core_trends), ""])- if result.keyword_analysis:- lines.extend(["[热度走势]", result.keyword_analysis, ""])-- if result.sentiment:- lines.extend(["[情感倾向]", result.sentiment, ""])-- if result.cross_platform:- lines.extend(["[跨平台关联]", result.cross_platform, ""])-- if result.impact:- lines.extend(["[潜在影响]", result.impact, ""])+ if result.sentiment_controversy:+ lines.extend(["[舆论风向争议]", _format_list_content(result.sentiment_controversy), ""])if result.signals:- lines.extend(["[值得关注]", result.signals, ""])+ lines.extend(["[异动与弱信号]", _format_list_content(result.signals), ""])- if result.conclusion:- lines.extend(["[总结建议]", result.conclusion])+ if result.rss_insights:+ lines.extend(["[RSS 深度洞察]", _format_list_content(result.rss_insights), ""])++ if result.outlook_strategy:+ lines.extend(["[研判策略建议]", _format_list_content(result.outlook_strategy)])return "\n".join(lines)@@ -212,9 +223,79 @@ def get_ai_analysis_renderer(channel: str):"dingtalk": render_ai_analysis_dingtalk,"wework": render_ai_analysis_markdown,"telegram": render_ai_analysis_markdown,- "email": render_ai_analysis_html,+ "email": render_ai_analysis_html_rich, # 邮件使用丰富样式,配合 HTML 报告的 CSS"ntfy": render_ai_analysis_markdown,"bark": render_ai_analysis_plain,"slack": render_ai_analysis_markdown,}return renderers.get(channel, render_ai_analysis_markdown)+++def render_ai_analysis_html_rich(result: AIAnalysisResult) -> str:+ """渲染为丰富样式的 HTML 格式(HTML 报告用)"""+ if not result:+ return ""++ # 检查是否成功+ if not result.success:+ error_msg = result.error or "未知错误"+ return f'''+ <div class="ai-section">+ <div class="ai-error">⚠️ AI 分析失败: {_escape_html(str(error_msg))}</div>+ </div>'''++ ai_html = '''+ <div class="ai-section">+ <div class="ai-section-header">+ <div class="ai-section-title">✨ AI 热点分析</div>+ <span class="ai-section-badge">AI</span>+ </div>'''++ if result.core_trends:+ content = _format_list_content(result.core_trends)+ content_html = _escape_html(content).replace("\n", "<br>")+ ai_html += f'''+ <div class="ai-block">+ <div class="ai-block-title">核心热点态势</div>+ <div class="ai-block-content">{content_html}</div>+ </div>'''++ if result.sentiment_controversy:+ content = _format_list_content(result.sentiment_controversy)+ content_html = _escape_html(content).replace("\n", "<br>")+ ai_html += f'''+ <div class="ai-block">+ <div class="ai-block-title">舆论风向争议</div>+ <div class="ai-block-content">{content_html}</div>+ </div>'''++ if result.signals:+ content = _format_list_content(result.signals)+ content_html = _escape_html(content).replace("\n", "<br>")+ ai_html += f'''+ <div class="ai-block">+ <div class="ai-block-title">异动与弱信号</div>+ <div class="ai-block-content">{content_html}</div>+ </div>'''++ if result.rss_insights:+ content = _format_list_content(result.rss_insights)+ content_html = _escape_html(content).replace("\n", "<br>")+ ai_html += f'''+ <div class="ai-block">+ <div class="ai-block-title">RSS 深度洞察</div>+ <div class="ai-block-content">{content_html}</div>+ </div>'''++ if result.outlook_strategy:+ content = _format_list_content(result.outlook_strategy)+ content_html = _escape_html(content).replace("\n", "<br>")+ ai_html += f'''+ <div class="ai-block">+ <div class="ai-block-title">研判策略建议</div>+ <div class="ai-block-content">{content_html}</div>+ </div>'''++ ai_html += '''+ </div>'''+ return ai_htmldiff --git a/trendradar/ai/translator.py b/trendradar/ai/translator.pynew file mode 100644index 0000000000..1600a58c96--- /dev/null+++ b/trendradar/ai/translator.py@@ -0,0 +1,428 @@+# coding=utf-8+"""+AI 翻译器模块++对推送内容进行多语言翻译+使用共享的 AI 模型配置+"""++import json+import os+from dataclasses import dataclass, field+from pathlib import Path+from typing import Any, Dict, List, Optional+++@dataclass+class TranslationResult:+ """翻译结果"""+ translated_text: str = "" # 翻译后的文本+ original_text: str = "" # 原始文本+ success: bool = False # 是否成功+ error: str = "" # 错误信息+++@dataclass+class BatchTranslationResult:+ """批量翻译结果"""+ results: List[TranslationResult] = field(default_factory=list)+ success_count: int = 0+ fail_count: int = 0+ total_count: int = 0+++class AITranslator:+ """AI 翻译器"""++ def __init__(self, translation_config: Dict[str, Any], ai_config: Dict[str, Any]):+ """+ 初始化 AI 翻译器++ Args:+ translation_config: AI 翻译配置 (AI_TRANSLATION)+ ai_config: AI 模型共享配置 (AI)+ """+ self.translation_config = translation_config+ self.ai_config = ai_config++ # 翻译配置+ self.enabled = translation_config.get("ENABLED", False)+ self.target_language = translation_config.get("LANGUAGE", "English")++ # 从共享配置获取模型参数+ self.api_key = ai_config.get("API_KEY") or os.environ.get("AI_API_KEY", "")+ self.provider = ai_config.get("PROVIDER", "deepseek")+ self.model = ai_config.get("MODEL", "deepseek-chat")+ self.base_url = ai_config.get("BASE_URL", "")+ self.timeout = ai_config.get("TIMEOUT", 90)++ # AI 参数配置+ self.temperature = ai_config.get("TEMPERATURE", 1.0)+ self.max_tokens = ai_config.get("MAX_TOKENS", 5000)++ # 额外参数+ self.extra_params = ai_config.get("EXTRA_PARAMS", {})+ if isinstance(self.extra_params, str) and self.extra_params.strip():+ try:+ self.extra_params = json.loads(self.extra_params)+ except json.JSONDecodeError:+ print(f"[翻译] 解析 extra_params 失败,将忽略: {self.extra_params}")+ self.extra_params = {}++ if not isinstance(self.extra_params, dict):+ self.extra_params = {}++ # 加载提示词模板+ self.system_prompt, self.user_prompt_template = self._load_prompt_template(+ translation_config.get("PROMPT_FILE", "ai_translation_prompt.txt")+ )++ def _load_prompt_template(self, prompt_file: str) -> tuple:+ """加载提示词模板"""+ config_dir = Path(__file__).parent.parent.parent / "config"+ prompt_path = config_dir / prompt_file++ if not prompt_path.exists():+ print(f"[翻译] 提示词文件不存在: {prompt_path}")+ return "", ""++ content = prompt_path.read_text(encoding="utf-8")++ # 解析 [system] 和 [user] 部分+ system_prompt = ""+ user_prompt = ""++ if "[system]" in content and "[user]" in content:+ parts = content.split("[user]")+ system_part = parts[0]+ user_part = parts[1] if len(parts) > 1 else ""++ if "[system]" in system_part:+ system_prompt = system_part.split("[system]")[1].strip()++ user_prompt = user_part.strip()+ else:+ user_prompt = content++ return system_prompt, user_prompt++ def translate(self, text: str) -> TranslationResult:+ """+ 翻译单条文本++ Args:+ text: 要翻译的文本++ Returns:+ TranslationResult: 翻译结果+ """+ result = TranslationResult(original_text=text)++ if not self.enabled:+ result.error = "翻译功能未启用"+ return result++ if not self.api_key:+ result.error = "未配置 AI API Key"+ return result++ if not text or not text.strip():+ result.translated_text = text+ result.success = True+ return result++ try:+ # 构建提示词+ user_prompt = self.user_prompt_template+ user_prompt = user_prompt.replace("{target_language}", self.target_language)+ user_prompt = user_prompt.replace("{content}", text)++ # 调用 AI API+ response = self._call_ai_api(user_prompt)+ result.translated_text = response.strip()+ result.success = True++ except Exception as e:+ import requests+ error_type = type(e).__name__+ error_msg = str(e)++ if isinstance(e, requests.exceptions.Timeout):+ result.error = f"翻译请求超时({self.timeout}秒)"+ elif isinstance(e, requests.exceptions.ConnectionError):+ result.error = f"无法连接到 AI API"+ elif isinstance(e, requests.exceptions.HTTPError):+ status_code = e.response.status_code if hasattr(e, 'response') and e.response else "未知"+ if status_code == 401:+ result.error = "API 认证失败"+ elif status_code == 429:+ result.error = "API 请求频率过高"+ else:+ result.error = f"API 错误 (HTTP {status_code})"+ else:+ if len(error_msg) > 100:+ error_msg = error_msg[:100] + "..."+ result.error = f"翻译失败 ({error_type}): {error_msg}"++ return result++ def translate_batch(self, texts: List[str]) -> BatchTranslationResult:+ """+ 批量翻译文本(单次 API 调用)++ Args:+ texts: 要翻译的文本列表++ Returns:+ BatchTranslationResult: 批量翻译结果+ """+ batch_result = BatchTranslationResult(total_count=len(texts))++ if not self.enabled:+ for text in texts:+ batch_result.results.append(TranslationResult(+ original_text=text,+ error="翻译功能未启用"+ ))+ batch_result.fail_count = len(texts)+ return batch_result++ if not self.api_key:+ for text in texts:+ batch_result.results.append(TranslationResult(+ original_text=text,+ error="未配置 AI API Key"+ ))+ batch_result.fail_count = len(texts)+ return batch_result++ if not texts:+ return batch_result++ # 过滤空文本+ non_empty_indices = []+ non_empty_texts = []+ for i, text in enumerate(texts):+ if text and text.strip():+ non_empty_indices.append(i)+ non_empty_texts.append(text)++ # 初始化结果列表+ for text in texts:+ batch_result.results.append(TranslationResult(original_text=text))++ # 空文本直接标记成功+ for i, text in enumerate(texts):+ if not text or not text.strip():+ batch_result.results[i].translated_text = text+ batch_result.results[i].success = True+ batch_result.success_count += 1++ if not non_empty_texts:+ return batch_result++ try:+ # 构建批量翻译内容(使用编号格式)+ batch_content = self._format_batch_content(non_empty_texts)++ # 构建提示词+ user_prompt = self.user_prompt_template+ user_prompt = user_prompt.replace("{target_language}", self.target_language)+ user_prompt = user_prompt.replace("{content}", batch_content)++ # 调用 AI API+ response = self._call_ai_api(user_prompt)++ # 解析批量翻译结果+ translated_texts = self._parse_batch_response(response, len(non_empty_texts))++ # 填充结果+ for idx, translated in zip(non_empty_indices, translated_texts):+ batch_result.results[idx].translated_text = translated+ batch_result.results[idx].success = True+ batch_result.success_count += 1++ except Exception as e:+ error_msg = f"批量翻译失败: {type(e).__name__}: {str(e)[:100]}"+ for idx in non_empty_indices:+ batch_result.results[idx].error = error_msg+ batch_result.fail_count = len(non_empty_indices)++ return batch_result++ def _format_batch_content(self, texts: List[str]) -> str:+ """格式化批量翻译内容"""+ lines = []+ for i, text in enumerate(texts, 1):+ lines.append(f"[{i}] {text}")+ return "\n".join(lines)++ def _parse_batch_response(self, response: str, expected_count: int) -> List[str]:+ """+ 解析批量翻译响应++ Args:+ response: AI 响应文本+ expected_count: 期望的翻译数量++ Returns:+ List[str]: 翻译结果列表+ """+ results = []+ lines = response.strip().split("\n")++ current_idx = None+ current_text = []++ for line in lines:+ # 尝试匹配 [数字] 格式+ stripped = line.strip()+ if stripped.startswith("[") and "]" in stripped:+ bracket_end = stripped.index("]")+ try:+ idx = int(stripped[1:bracket_end])+ # 保存之前的内容+ if current_idx is not None:+ results.append((current_idx, "\n".join(current_text).strip()))+ current_idx = idx+ current_text = [stripped[bracket_end + 1:].strip()]+ except ValueError:+ if current_idx is not None:+ current_text.append(line)+ else:+ if current_idx is not None:+ current_text.append(line)++ # 保存最后一条+ if current_idx is not None:+ results.append((current_idx, "\n".join(current_text).strip()))++ # 按索引排序并提取文本+ results.sort(key=lambda x: x[0])+ translated = [text for _, text in results]++ # 如果解析结果数量不匹配,尝试简单按行分割+ if len(translated) != expected_count:+ # 回退:按行分割(去除编号)+ translated = []+ for line in lines:+ stripped = line.strip()+ if stripped.startswith("[") and "]" in stripped:+ bracket_end = stripped.index("]")+ translated.append(stripped[bracket_end + 1:].strip())+ elif stripped:+ translated.append(stripped)++ # 确保返回正确数量+ while len(translated) < expected_count:+ translated.append("")++ return translated[:expected_count]++ def _call_ai_api(self, user_prompt: str) -> str:+ """调用 AI API"""+ if self.provider == "gemini":+ return self._call_gemini(user_prompt)+ return self._call_openai_compatible(user_prompt)++ def _get_api_url(self) -> str:+ """获取完整 API URL"""+ if self.base_url:+ return self.base_url++ urls = {+ "deepseek": "https://api.deepseek.com/v1/chat/completions",+ "openai": "https://api.openai.com/v1/chat/completions",+ }+ url = urls.get(self.provider)+ if not url:+ raise ValueError(f"{self.provider} 需要配置 base_url")+ return url++ def _call_openai_compatible(self, user_prompt: str) -> str:+ """调用 OpenAI 兼容接口"""+ import requests++ url = self._get_api_url()++ headers = {+ "Authorization": f"Bearer {self.api_key}",+ "Content-Type": "application/json",+ }++ messages = []+ if self.system_prompt:+ messages.append({"role": "system", "content": self.system_prompt})+ messages.append({"role": "user", "content": user_prompt})++ payload = {+ "model": self.model,+ "messages": messages,+ "temperature": self.temperature,+ }++ if self.max_tokens:+ payload["max_tokens"] = self.max_tokens++ if self.extra_params:+ payload.update(self.extra_params)++ response = requests.post(+ url,+ headers=headers,+ json=payload,+ timeout=self.timeout,+ )+ response.raise_for_status()++ data = response.json()+ return data["choices"][0]["message"]["content"]++ def _call_gemini(self, user_prompt: str) -> str:+ """调用 Google Gemini API"""+ import requests++ model = self.model or "gemini-1.5-flash"+ url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={self.api_key}"++ headers = {+ "Content-Type": "application/json",+ }++ payload = {+ "contents": [{+ "role": "user",+ "parts": [{"text": user_prompt}]+ }],+ "generationConfig": {+ "temperature": self.temperature,+ },+ "safetySettings": [+ {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},+ {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"},+ ]+ }++ if self.system_prompt:+ payload["system_instruction"] = {+ "parts": [{"text": self.system_prompt}]+ }++ if self.max_tokens:+ payload["generationConfig"]["maxOutputTokens"] = self.max_tokens++ if self.extra_params:+ payload["generationConfig"].update(self.extra_params)++ response = requests.post(+ url,+ headers=headers,+ json=payload,+ timeout=self.timeout,+ )+ response.raise_for_status()++ data = response.json()+ return data["candidates"][0]["content"]["parts"][0]["text"]diff --git a/trendradar/context.py b/trendradar/context.pyindex c48201c7ae..c08db86e1a 100644--- a/trendradar/context.py+++ b/trendradar/context.py@@ -7,7 +7,7 @@from datetime import datetimefrom pathlib import Path-from typing import Any, Callable, Dict, List, Optional, Tuple+from typing import Any, Dict, List, Optional, Tuplefrom trendradar.utils.time import (get_configured_time,@@ -22,7 +22,6 @@ from trendradar.core import (save_titles_to_file,read_all_today_titles,detect_latest_new_titles,- is_first_crawl_today,count_word_frequency,)from trendradar.report import (@@ -38,6 +37,7 @@ from trendradar.notification import (NotificationDispatcher,PushRecordManager,)+from trendradar.ai import AITranslatorfrom trendradar.storage import get_storage_manager@@ -120,6 +120,17 @@ class AppContext:"""获取显示模式 (keyword | platform)"""return self.config.get("DISPLAY_MODE", "keyword")+ @property+ def show_new_section(self) -> bool:+ """是否显示新增热点区域"""+ return self.config.get("DISPLAY", {}).get("REGIONS", {}).get("NEW_ITEMS", True)++ @property+ def region_order(self) -> List[str]:+ """获取区域显示顺序"""+ default_order = ["hotlist", "rss", "new_items", "standalone", "ai_analysis"]+ return self.config.get("DISPLAY", {}).get("REGION_ORDER", default_order)+# === 时间操作 ===def get_time(self) -> datetime:@@ -174,8 +185,8 @@ class AppContext:return self._storage_managerdef get_output_path(self, subfolder: str, filename: str) -> str:- """获取输出路径"""- output_dir = Path("output") / self.format_date() / subfolder+ """获取输出路径(扁平化结构:output/类型/日期/文件名)"""+ output_dir = Path("output") / subfolder / self.format_date()output_dir.mkdir(parents=True, exist_ok=True)return str(output_dir / filename)@@ -273,6 +284,7 @@ class AppContext:rank_threshold=self.rank_threshold,matches_word_groups_func=self.matches_word_groups,load_frequency_words_func=self.load_frequency_words,+ show_new_section=self.show_new_section,)def generate_html(@@ -283,10 +295,11 @@ class AppContext:new_titles: Optional[Dict] = None,id_to_name: Optional[Dict] = None,mode: str = "daily",- is_daily_summary: bool = False,update_info: Optional[Dict] = None,rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,+ ai_analysis: Optional[Any] = None,+ standalone_data: Optional[Dict] = None,) -> str:"""生成HTML报告"""return generate_html_report(@@ -296,40 +309,41 @@ class AppContext:new_titles=new_titles,id_to_name=id_to_name,mode=mode,- is_daily_summary=is_daily_summary,update_info=update_info,rank_threshold=self.rank_threshold,output_dir="output",date_folder=self.format_date(),time_filename=self.format_time(),- render_html_func=lambda *args, **kwargs: self.render_html(*args, rss_items=rss_items, rss_new_items=rss_new_items, **kwargs),+ render_html_func=lambda *args, **kwargs: self.render_html(*args, rss_items=rss_items, rss_new_items=rss_new_items, ai_analysis=ai_analysis, standalone_data=standalone_data, **kwargs),matches_word_groups_func=self.matches_word_groups,load_frequency_words_func=self.load_frequency_words,- enable_index_copy=True,)def render_html(self,report_data: Dict,total_titles: int,- is_daily_summary: bool = False,mode: str = "daily",update_info: Optional[Dict] = None,rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,+ ai_analysis: Optional[Any] = None,+ standalone_data: Optional[Dict] = None,) -> str:"""渲染HTML内容"""return render_html_content(report_data=report_data,total_titles=total_titles,- is_daily_summary=is_daily_summary,mode=mode,update_info=update_info,- reverse_content_order=self.config.get("REVERSE_CONTENT_ORDER", False),+ region_order=self.region_order,get_time_func=self.get_time,rss_items=rss_items,rss_new_items=rss_new_items,display_mode=self.display_mode,+ ai_analysis=ai_analysis,+ show_new_section=self.show_new_section,+ standalone_data=standalone_data,)# === 通知内容渲染 ===@@ -346,8 +360,9 @@ class AppContext:update_info=update_info,mode=mode,separator=self.config.get("FEISHU_MESSAGE_SEPARATOR", "---"),- reverse_content_order=self.config.get("REVERSE_CONTENT_ORDER", False),+ region_order=self.region_order,get_time_func=self.get_time,+ show_new_section=self.show_new_section,)def render_dingtalk(@@ -361,8 +376,9 @@ class AppContext:report_data=report_data,update_info=update_info,mode=mode,- reverse_content_order=self.config.get("REVERSE_CONTENT_ORDER", False),+ region_order=self.region_order,get_time_func=self.get_time,+ show_new_section=self.show_new_section,)def split_content(@@ -409,7 +425,7 @@ class AppContext:"default": self.config.get("MESSAGE_BATCH_SIZE", 4000),},feishu_separator=self.config.get("FEISHU_MESSAGE_SEPARATOR", "---"),- reverse_content_order=self.config.get("REVERSE_CONTENT_ORDER", False),+ region_order=self.region_order,get_time_func=self.get_time,rss_items=rss_items,rss_new_items=rss_new_items,@@ -420,16 +436,25 @@ class AppContext:rank_threshold=self.rank_threshold,ai_stats=ai_stats,report_type=report_type,+ show_new_section=self.show_new_section,)# === 通知发送 ===def create_notification_dispatcher(self) -> NotificationDispatcher:"""创建通知调度器"""+ # 创建翻译器(如果启用)+ translator = None+ trans_config = self.config.get("AI_TRANSLATION", {})+ if trans_config.get("ENABLED", False):+ ai_config = self.config.get("AI", {})+ translator = AITranslator(trans_config, ai_config)+return NotificationDispatcher(config=self.config,get_time_func=self.get_time,split_content_func=self.split_content,+ translator=translator,)def create_push_manager(self) -> PushRecordManager:diff --git a/trendradar/core/__init__.py b/trendradar/core/__init__.pyindex d4af0d354d..2eb9305881 100644--- a/trendradar/core/__init__.py+++ b/trendradar/core/__init__.py@@ -17,7 +17,6 @@ from trendradar.core.data import (read_all_today_titles,detect_latest_new_titles_from_storage,detect_latest_new_titles,- is_first_crawl_today,)from trendradar.core.analyzer import (calculate_news_weight,@@ -40,7 +39,6 @@ __all__ = ["read_all_today_titles","detect_latest_new_titles_from_storage","detect_latest_new_titles",- "is_first_crawl_today",# 统计分析"calculate_news_weight","format_time_display",diff --git a/trendradar/core/analyzer.py b/trendradar/core/analyzer.pyindex 6eac8cb566..2bf9dff0ca 100644--- a/trendradar/core/analyzer.py+++ b/trendradar/core/analyzer.py@@ -290,6 +290,7 @@ def count_word_frequency(ranks = source_ranks if source_ranks else []url = source_urlmobile_url = source_mobile_url+ rank_timeline = []# 对于 current 模式,从历史统计信息中获取完整数据if (@@ -306,6 +307,7 @@ def count_word_frequency(ranks = info["ranks"]url = info.get("url", source_url)mobile_url = info.get("mobileUrl", source_mobile_url)+ rank_timeline = info.get("rank_timeline", [])elif (title_infoand source_id in title_info@@ -319,6 +321,7 @@ def count_word_frequency(ranks = info["ranks"]url = info.get("url", source_url)mobile_url = info.get("mobileUrl", source_mobile_url)+ rank_timeline = info.get("rank_timeline", [])if not ranks:ranks = [99]@@ -350,6 +353,7 @@ def count_word_frequency("url": url,"mobileUrl": mobile_url,"is_new": is_new,+ "rank_timeline": rank_timeline,})diff --git a/trendradar/core/data.py b/trendradar/core/data.pyindex 7297e8897d..ba1b7d5928 100644--- a/trendradar/core/data.py+++ b/trendradar/core/data.py@@ -126,6 +126,7 @@ def read_all_today_titles_from_storage(first_time = getattr(item, 'first_time', item.crawl_time)last_time = getattr(item, 'last_time', item.crawl_time)count = getattr(item, 'count', 1)+ rank_timeline = getattr(item, 'rank_timeline', [])all_results[source_id][title] = {"ranks": ranks,@@ -140,6 +141,7 @@ def read_all_today_titles_from_storage("ranks": ranks,"url": item.url or "","mobileUrl": item.mobile_url or "",+ "rank_timeline": rank_timeline,}return all_results, final_id_to_name, title_info@@ -283,23 +285,3 @@ def detect_latest_new_titles(total_new = sum(len(titles) for titles in new_titles.values())print(f"[存储] 从存储后端检测到 {total_new} 条新增标题")return new_titles---def is_first_crawl_today(output_dir: str, date_folder: str) -> bool:- """- 检测是否是当天第一次爬取-- Args:- output_dir: 输出目录- date_folder: 日期文件夹名称-- Returns:- bool: 是否是当天第一次爬取- """- txt_dir = Path(output_dir) / date_folder / "txt"-- if not txt_dir.exists():- return True-- files = sorted([f for f in txt_dir.iterdir() if f.suffix == ".txt"])- return len(files) <= 1diff --git a/trendradar/core/loader.py b/trendradar/core/loader.pyindex 62b65840d8..5fdd0d5077 100644--- a/trendradar/core/loader.py+++ b/trendradar/core/loader.py@@ -65,12 +65,12 @@ def _load_crawler_config(config_data: Dict) -> Dict:"""加载爬虫配置"""advanced = config_data.get("advanced", {})crawler_config = advanced.get("crawler", {})- enable_crawler_env = _get_env_bool("ENABLE_CRAWLER")+ platforms_config = config_data.get("platforms", {})return {"REQUEST_INTERVAL": crawler_config.get("request_interval", 100),"USE_PROXY": crawler_config.get("use_proxy", False),"DEFAULT_PROXY": crawler_config.get("default_proxy", ""),- "ENABLE_CRAWLER": enable_crawler_env if enable_crawler_env is not None else crawler_config.get("enabled", True),+ "ENABLE_CRAWLER": platforms_config.get("enabled", True),}@@ -80,17 +80,14 @@ def _load_report_config(config_data: Dict) -> Dict:# 环境变量覆盖sort_by_position_env = _get_env_bool("SORT_BY_POSITION_FIRST")- reverse_content_env = _get_env_bool("REVERSE_CONTENT_ORDER")max_news_env = _get_env_int("MAX_NEWS_PER_KEYWORD")- display_mode_env = _get_env_str("DISPLAY_MODE")return {- "REPORT_MODE": _get_env_str("REPORT_MODE") or report_config.get("mode", "daily"),- "DISPLAY_MODE": display_mode_env or report_config.get("display_mode", "keyword"),+ "REPORT_MODE": report_config.get("mode", "daily"),+ "DISPLAY_MODE": report_config.get("display_mode", "keyword"),"RANK_THRESHOLD": report_config.get("rank_threshold", 10),"SORT_BY_POSITION_FIRST": sort_by_position_env if sort_by_position_env is not None else report_config.get("sort_by_position_first", False),"MAX_NEWS_PER_KEYWORD": max_news_env or report_config.get("max_news_per_keyword", 0),- "REVERSE_CONTENT_ORDER": reverse_content_env if reverse_content_env is not None else report_config.get("reverse_content_order", False),}@@ -100,10 +97,8 @@ def _load_notification_config(config_data: Dict) -> Dict:advanced = config_data.get("advanced", {})batch_size = advanced.get("batch_size", {})- enable_notification_env = _get_env_bool("ENABLE_NOTIFICATION")-return {- "ENABLE_NOTIFICATION": enable_notification_env if enable_notification_env is not None else notification.get("enabled", True),+ "ENABLE_NOTIFICATION": notification.get("enabled", True),"MESSAGE_BATCH_SIZE": batch_size.get("default", 4000),"DINGTALK_BATCH_SIZE": batch_size.get("dingtalk", 20000),"FEISHU_BATCH_SIZE": batch_size.get("feishu", 29000),@@ -180,43 +175,91 @@ def _load_rss_config(config_data: Dict) -> Dict:"ENABLED": freshness_filter.get("enabled", True), # 默认启用"MAX_AGE_DAYS": max_age_days,},- "NOTIFICATION": {- "ENABLED": advanced_rss.get("notification_enabled", False),- },}-def _load_standalone_display_config(config_data: Dict) -> Dict:- """加载独立展示区配置"""- notification = config_data.get("notification", {})- standalone = notification.get("standalone_display", {})+def _load_display_config(config_data: Dict) -> Dict:+ """加载推送内容显示配置"""+ display = config_data.get("display", {})+ regions = display.get("regions", {})+ standalone = display.get("standalone", {})++ # 默认区域顺序+ default_region_order = ["hotlist", "rss", "new_items", "standalone", "ai_analysis"]+ region_order = display.get("region_order", default_region_order)++ # 验证 region_order 中的值是否合法+ valid_regions = {"hotlist", "rss", "new_items", "standalone", "ai_analysis"}+ region_order = [r for r in region_order if r in valid_regions]++ # 如果过滤后为空,使用默认顺序+ if not region_order:+ region_order = default_region_orderreturn {- "ENABLED": standalone.get("enabled", False),- "PLATFORMS": standalone.get("platforms", []),- "RSS_FEEDS": standalone.get("rss_feeds", []),- "MAX_ITEMS": standalone.get("max_items", 20),+ # 区域显示顺序+ "REGION_ORDER": region_order,+ # 区域开关+ "REGIONS": {+ "HOTLIST": regions.get("hotlist", True),+ "NEW_ITEMS": regions.get("new_items", True),+ "RSS": regions.get("rss", True),+ "STANDALONE": regions.get("standalone", False),+ "AI_ANALYSIS": regions.get("ai_analysis", True),+ },+ # 独立展示区配置+ "STANDALONE": {+ "PLATFORMS": standalone.get("platforms", []),+ "RSS_FEEDS": standalone.get("rss_feeds", []),+ "MAX_ITEMS": standalone.get("max_items", 20),+ },}-def _load_ai_analysis_config(config_data: Dict) -> Dict:- """加载 AI 分析配置"""- ai_config = config_data.get("ai_analysis", {})+def _load_ai_config(config_data: Dict) -> Dict:+ """加载 AI 模型共享配置"""+ ai_config = config_data.get("ai", {})- enabled_env = _get_env_bool("AI_ANALYSIS_ENABLED")timeout_env = _get_env_int_or_none("AI_TIMEOUT")return {- "ENABLED": enabled_env if enabled_env is not None else ai_config.get("enabled", False),"PROVIDER": _get_env_str("AI_PROVIDER") or ai_config.get("provider", "deepseek"),"API_KEY": _get_env_str("AI_API_KEY") or ai_config.get("api_key", ""),"MODEL": _get_env_str("AI_MODEL") or ai_config.get("model", "deepseek-chat"),"BASE_URL": _get_env_str("AI_BASE_URL") or ai_config.get("base_url", ""),"TIMEOUT": timeout_env if timeout_env is not None else ai_config.get("timeout", 90),- "PUSH_MODE": _get_env_str("AI_PUSH_MODE") or ai_config.get("push_mode", "both"),+ "TEMPERATURE": ai_config.get("temperature", 1.0),+ "MAX_TOKENS": ai_config.get("max_tokens", 5000),+ "EXTRA_PARAMS": ai_config.get("extra_params", {}),+ }+++def _load_ai_analysis_config(config_data: Dict) -> Dict:+ """加载 AI 分析配置(功能配置,模型配置见 _load_ai_config)"""+ ai_config = config_data.get("ai_analysis", {})++ enabled_env = _get_env_bool("AI_ANALYSIS_ENABLED")++ return {+ "ENABLED": enabled_env if enabled_env is not None else ai_config.get("enabled", False),+ "LANGUAGE": ai_config.get("language", "Chinese"),+ "PROMPT_FILE": ai_config.get("prompt_file", "ai_analysis_prompt.txt"),"MAX_NEWS_FOR_ANALYSIS": ai_config.get("max_news_for_analysis", 50),"INCLUDE_RSS": ai_config.get("include_rss", True),- "PROMPT_FILE": ai_config.get("prompt_file", "ai_analysis_prompt.txt"),+ "INCLUDE_RANK_TIMELINE": ai_config.get("include_rank_timeline", False),+ }+++def _load_ai_translation_config(config_data: Dict) -> Dict:+ """加载 AI 翻译配置(功能配置,模型配置见 _load_ai_config)"""+ trans_config = config_data.get("ai_translation", {})++ enabled_env = _get_env_bool("AI_TRANSLATION_ENABLED")++ return {+ "ENABLED": enabled_env if enabled_env is not None else trans_config.get("enabled", False),+ "LANGUAGE": _get_env_str("AI_TRANSLATION_LANGUAGE") or trans_config.get("language", "English"),+ "PROMPT_FILE": trans_config.get("prompt_file", "ai_translation_prompt.txt"),}@@ -300,8 +343,8 @@ def _load_webhook_config(config_data: Dict) -> Dict:# Slack"SLACK_WEBHOOK_URL": _get_env_str("SLACK_WEBHOOK_URL") or slack.get("webhook_url", ""),# 通用 Webhook- "GENERIC_WEBHOOK_URL": _get_env_str("GENERIC_WEBHOOK_URL") or generic.get("url", ""),- "GENERIC_WEBHOOK_TEMPLATE": _get_env_str("GENERIC_WEBHOOK_TEMPLATE") or generic.get("template", ""),+ "GENERIC_WEBHOOK_URL": _get_env_str("GENERIC_WEBHOOK_URL") or generic.get("webhook_url", ""),+ "GENERIC_WEBHOOK_TEMPLATE": _get_env_str("GENERIC_WEBHOOK_TEMPLATE") or generic.get("payload_template", ""),}@@ -433,16 +476,23 @@ def load_config(config_path: Optional[str] = None) -> Dict[str, Any]:config["WEIGHT_CONFIG"] = _load_weight_config(config_data)# 平台配置- config["PLATFORMS"] = config_data.get("platforms", [])+ platforms_config = config_data.get("platforms", {})+ config["PLATFORMS"] = platforms_config.get("sources", [])# RSS 配置config["RSS"] = _load_rss_config(config_data)+ # AI 模型共享配置+ config["AI"] = _load_ai_config(config_data)+# AI 分析配置config["AI_ANALYSIS"] = _load_ai_analysis_config(config_data)- # 独立展示区配置- config["STANDALONE_DISPLAY"] = _load_standalone_display_config(config_data)+ # AI 翻译配置+ config["AI_TRANSLATION"] = _load_ai_translation_config(config_data)++ # 推送内容显示配置+ config["DISPLAY"] = _load_display_config(config_data)# 存储配置config["STORAGE"] = _load_storage_config(config_data)diff --git a/trendradar/notification/dispatcher.py b/trendradar/notification/dispatcher.pyindex d2aca27157..0a0e6860a1 100644--- a/trendradar/notification/dispatcher.py+++ b/trendradar/notification/dispatcher.py@@ -40,7 +40,7 @@ from .renderer import (# 类型检查时导入,运行时不导入(避免循环导入)if TYPE_CHECKING:- from trendradar.ai import AIAnalysisResult+ from trendradar.ai import AIAnalysisResult, AITranslatorclass NotificationDispatcher:@@ -56,6 +56,7 @@ class NotificationDispatcher:config: Dict[str, Any],get_time_func: Callable,split_content_func: Callable,+ translator: Optional["AITranslator"] = None,):"""初始化通知调度器@@ -64,11 +65,99 @@ class NotificationDispatcher:config: 完整的配置字典,包含所有通知渠道的配置get_time_func: 获取当前时间的函数split_content_func: 内容分批函数+ translator: AI 翻译器实例(可选)"""self.config = configself.get_time_func = get_time_funcself.split_content_func = split_content_funcself.max_accounts = config.get("MAX_ACCOUNTS_PER_CHANNEL", 3)+ self.translator = translator++ def _translate_content(+ self,+ report_data: Dict,+ rss_items: Optional[List[Dict]] = None,+ rss_new_items: Optional[List[Dict]] = None,+ ) -> tuple:+ """+ 翻译推送内容++ Args:+ report_data: 报告数据+ rss_items: RSS 统计条目+ rss_new_items: RSS 新增条目++ Returns:+ tuple: (翻译后的 report_data, rss_items, rss_new_items)+ """+ if not self.translator or not self.translator.enabled:+ return report_data, rss_items, rss_new_items++ import copy+ print(f"[翻译] 开始翻译内容到 {self.translator.target_language}...")++ # 深拷贝避免修改原始数据+ report_data = copy.deepcopy(report_data)+ rss_items = copy.deepcopy(rss_items) if rss_items else None+ rss_new_items = copy.deepcopy(rss_new_items) if rss_new_items else None++ # 收集所有需要翻译的标题+ titles_to_translate = []+ title_locations = [] # 记录标题位置,用于回填++ # 1. 热榜标题+ for stat_idx, stat in enumerate(report_data.get("stats", [])):+ for title_idx, title_data in enumerate(stat.get("titles", [])):+ titles_to_translate.append(title_data.get("title", ""))+ title_locations.append(("stats", stat_idx, title_idx))++ # 2. 新增热点标题+ for source_idx, source in enumerate(report_data.get("new_titles", [])):+ for title_idx, title_data in enumerate(source.get("titles", [])):+ titles_to_translate.append(title_data.get("title", ""))+ title_locations.append(("new_titles", source_idx, title_idx))++ # 3. RSS 统计标题+ if rss_items:+ for item_idx, item in enumerate(rss_items):+ titles_to_translate.append(item.get("title", ""))+ title_locations.append(("rss_items", item_idx, None))++ # 4. RSS 新增标题+ if rss_new_items:+ for item_idx, item in enumerate(rss_new_items):+ titles_to_translate.append(item.get("title", ""))+ title_locations.append(("rss_new_items", item_idx, None))++ if not titles_to_translate:+ print("[翻译] 没有需要翻译的内容")+ return report_data, rss_items, rss_new_items++ print(f"[翻译] 共 {len(titles_to_translate)} 条标题待翻译")++ # 批量翻译+ result = self.translator.translate_batch(titles_to_translate)++ if result.success_count == 0:+ print(f"[翻译] 翻译失败: {result.results[0].error if result.results else '未知错误'}")+ return report_data, rss_items, rss_new_items++ print(f"[翻译] 翻译完成: {result.success_count}/{result.total_count} 成功")++ # 回填翻译结果+ for i, (loc_type, idx1, idx2) in enumerate(title_locations):+ if i < len(result.results) and result.results[i].success:+ translated = result.results[i].translated_text+ if loc_type == "stats":+ report_data["stats"][idx1]["titles"][idx2]["title"] = translated+ elif loc_type == "new_titles":+ report_data["new_titles"][idx1]["titles"][idx2]["title"] = translated+ elif loc_type == "rss_items" and rss_items:+ rss_items[idx1]["title"] = translated+ elif loc_type == "rss_new_items" and rss_new_items:+ rss_new_items[idx1]["title"] = translated++ return report_data, rss_items, rss_new_itemsdef dispatch_all(self,@@ -103,73 +192,77 @@ class NotificationDispatcher:"""results = {}- # 获取 AI 推送模式- ai_config = self.config.get("AI_ANALYSIS", {})- ai_push_mode = ai_config.get("PUSH_MODE", "both")+ # 获取区域显示配置+ display_regions = self.config.get("DISPLAY", {}).get("REGIONS", {})++ # 执行翻译(如果启用)+ report_data, rss_items, rss_new_items = self._translate_content(+ report_data, rss_items, rss_new_items+ )# 飞书if self.config.get("FEISHU_WEBHOOK_URL"):results["feishu"] = self._send_feishu(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# 钉钉if self.config.get("DINGTALK_WEBHOOK_URL"):results["dingtalk"] = self._send_dingtalk(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# 企业微信if self.config.get("WEWORK_WEBHOOK_URL"):results["wework"] = self._send_wework(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# Telegram(需要配对验证)if self.config.get("TELEGRAM_BOT_TOKEN") and self.config.get("TELEGRAM_CHAT_ID"):results["telegram"] = self._send_telegram(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# ntfy(需要配对验证)if self.config.get("NTFY_SERVER_URL") and self.config.get("NTFY_TOPIC"):results["ntfy"] = self._send_ntfy(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# Barkif self.config.get("BARK_URL"):results["bark"] = self._send_bark(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# Slackif self.config.get("SLACK_WEBHOOK_URL"):results["slack"] = self._send_slack(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)# 通用 Webhookif self.config.get("GENERIC_WEBHOOK_URL"):results["generic_webhook"] = self._send_generic_webhook(report_data, report_type, update_info, proxy_url, mode, rss_items, rss_new_items,- ai_analysis, ai_push_mode, standalone_data+ ai_analysis, display_regions, standalone_data)- # 邮件(保持原有逻辑,已支持多收件人)+ # 邮件(保持原有逻辑,已支持多收件人,AI 分析已嵌入 HTML)if (self.config.get("EMAIL_FROM")and self.config.get("EMAIL_PASSWORD")and self.config.get("EMAIL_TO")):- results["email"] = self._send_email(report_type, html_file_path, ai_analysis, ai_push_mode)+ results["email"] = self._send_email(report_type, html_file_path)return results@@ -217,13 +310,14 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到飞书(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- # 根据 AI 推送模式决定是否发送原始内容- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ # 根据区域开关决定是否发送对应内容+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}return self._send_to_multi_accounts(channel_name="飞书",@@ -240,11 +334,11 @@ class NotificationDispatcher:batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,get_time_func=self.get_time_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,),)@@ -258,12 +352,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到钉钉(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}return self._send_to_multi_accounts(channel_name="钉钉",@@ -279,11 +374,11 @@ class NotificationDispatcher:batch_size=self.config.get("DINGTALK_BATCH_SIZE", 20000),batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,),)@@ -297,12 +392,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到企业微信(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}return self._send_to_multi_accounts(channel_name="企业微信",@@ -319,11 +415,11 @@ class NotificationDispatcher:batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),msg_type=self.config.get("WEWORK_MSG_TYPE", "markdown"),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,),)@@ -337,12 +433,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到 Telegram(多账号,需验证 token 和 chat_id 配对,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}telegram_tokens = parse_multi_account_config(self.config["TELEGRAM_BOT_TOKEN"])telegram_chat_ids = parse_multi_account_config(self.config["TELEGRAM_CHAT_ID"])@@ -381,11 +478,11 @@ class NotificationDispatcher:batch_size=self.config.get("MESSAGE_BATCH_SIZE", 4000),batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,)results.append(result)@@ -401,12 +498,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到 ntfy(多账号,需验证 topic 和 token 配对,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}ntfy_server_url = self.config["NTFY_SERVER_URL"]ntfy_topics = parse_multi_account_config(self.config["NTFY_TOPIC"])@@ -444,11 +542,11 @@ class NotificationDispatcher:account_label=account_label,batch_size=3800,split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,)results.append(result)@@ -464,12 +562,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到 Bark(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}return self._send_to_multi_accounts(channel_name="Bark",@@ -485,11 +584,11 @@ class NotificationDispatcher:batch_size=self.config.get("BARK_BATCH_SIZE", 3600),batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,),)@@ -503,12 +602,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到 Slack(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}return self._send_to_multi_accounts(channel_name="Slack",@@ -524,11 +624,11 @@ class NotificationDispatcher:batch_size=self.config.get("SLACK_BATCH_SIZE", 4000),batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,),)@@ -542,12 +642,13 @@ class NotificationDispatcher:rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""发送到通用 Webhook(多账号,支持热榜+RSS合并+AI分析+独立展示区)"""- if ai_push_mode == "only_analysis" and ai_analysis:- report_data = {"stats": [], "failed_ids": [], "new_titles": {}, "id_to_name": {}}+ display_regions = display_regions or {}+ if not display_regions.get("HOTLIST", True):+ report_data = {"stats": [], "failed_ids": [], "new_titles": [], "id_to_name": {}}urls = parse_multi_account_config(self.config.get("GENERIC_WEBHOOK_URL", ""))templates = parse_multi_account_config(self.config.get("GENERIC_WEBHOOK_TEMPLATE", ""))@@ -583,11 +684,11 @@ class NotificationDispatcher:batch_size=self.config.get("MESSAGE_BATCH_SIZE", 4000),batch_interval=self.config.get("BATCH_SEND_INTERVAL", 1.0),split_content_func=self.split_content_func,- rss_items=rss_items if ai_push_mode != "only_analysis" else None,- rss_new_items=rss_new_items if ai_push_mode != "only_analysis" else None,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,- standalone_data=standalone_data,+ rss_items=rss_items if display_regions.get("RSS", True) else None,+ rss_new_items=rss_new_items if display_regions.get("RSS", True) else None,+ ai_analysis=ai_analysis if display_regions.get("AI_ANALYSIS", True) else None,+ display_regions=display_regions,+ standalone_data=standalone_data if display_regions.get("STANDALONE", False) else None,)results.append(result)@@ -597,10 +698,12 @@ class NotificationDispatcher:self,report_type: str,html_file_path: Optional[str],- ai_analysis: Optional[AIAnalysisResult] = None,- ai_push_mode: str = "both",) -> bool:- """发送邮件(保持原有逻辑,已支持多收件人,支持AI分析)"""+ """发送邮件(保持原有逻辑,已支持多收件人)++ Note:+ AI 分析内容已在 HTML 生成时嵌入,无需在此传递+ """return send_to_email(from_email=self.config["EMAIL_FROM"],password=self.config["EMAIL_PASSWORD"],@@ -610,8 +713,6 @@ class NotificationDispatcher:custom_smtp_server=self.config.get("EMAIL_SMTP_SERVER", ""),custom_smtp_port=self.config.get("EMAIL_SMTP_PORT", ""),get_time_func=self.get_time_func,- ai_analysis=ai_analysis,- ai_push_mode=ai_push_mode,)# === RSS 通知方法 ===diff --git a/trendradar/notification/push_manager.py b/trendradar/notification/push_manager.pyindex 26a3da5421..f153e39a7f 100644--- a/trendradar/notification/push_manager.py+++ b/trendradar/notification/push_manager.py@@ -41,8 +41,9 @@ class PushRecordManager:print(f"[推送记录] 使用 {storage_backend.backend_name} 存储后端")def _default_get_time(self) -> datetime:- """默认时间获取函数(UTC+8)"""- return datetime.now(pytz.timezone("Asia/Shanghai"))+ """默认时间获取函数(使用 storage_backend 的时区配置)"""+ timezone = getattr(self.storage_backend, 'timezone', 'Asia/Shanghai')+ return datetime.now(pytz.timezone(timezone))def has_pushed_today(self) -> bool:"""diff --git a/trendradar/notification/renderer.py b/trendradar/notification/renderer.pyindex 065f78adfc..9f63b9df02 100644--- a/trendradar/notification/renderer.py+++ b/trendradar/notification/renderer.py@@ -6,19 +6,24 @@"""from datetime import datetime-from typing import Dict, Optional, Callable+from typing import Dict, List, Optional, Callablefrom trendradar.report.formatter import format_title_for_platform+# 默认区域顺序+DEFAULT_REGION_ORDER = ["hotlist", "rss", "new_items", "standalone", "ai_analysis"]++def render_feishu_content(report_data: Dict,update_info: Optional[Dict] = None,mode: str = "daily",separator: str = "---",- reverse_content_order: bool = False,+ region_order: Optional[List[str]] = None,get_time_func: Optional[Callable[[], datetime]] = None,rss_items: Optional[list] = None,+ show_new_section: bool = True,) -> str:"""渲染飞书通知内容(支持热榜+RSS合并)@@ -27,13 +32,17 @@ def render_feishu_content(update_info: 版本更新信息(可选)mode: 报告模式 ("daily", "incremental", "current")separator: 内容分隔符- reverse_content_order: 是否反转内容顺序(新增在前)+ region_order: 区域显示顺序列表get_time_func: 获取当前时间的函数(可选,默认使用 datetime.now())rss_items: RSS 条目列表(可选,用于合并推送)+ show_new_section: 是否显示新增热点区域Returns:格式化的飞书消息内容"""+ if region_order is None:+ region_order = DEFAULT_REGION_ORDER+# 生成热点词汇统计部分stats_content = ""if report_data["stats"]:@@ -68,7 +77,7 @@ def render_feishu_content(# 生成新增新闻部分new_titles_content = ""- if report_data["new_titles"]:+ if show_new_section and report_data["new_titles"]:new_titles_content += (f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n")@@ -88,31 +97,26 @@ def render_feishu_content(new_titles_content += "\n"- # 根据配置决定内容顺序- text_content = ""- if reverse_content_order:- # 新增热点在前,热点词汇统计在后- if new_titles_content:- text_content += new_titles_content- if stats_content:- text_content += f"\n{separator}\n\n"- if stats_content:- text_content += stats_content- else:- # 默认:热点词汇统计在前,新增热点在后- if stats_content:- text_content += stats_content- if new_titles_content:- text_content += f"\n{separator}\n\n"- if new_titles_content:- text_content += new_titles_content-- # 添加 RSS 内容(如果有)+ # RSS 内容+ rss_content = ""if rss_items:rss_content = _render_rss_section_feishu(rss_items, separator)- if text_content:- text_content += f"\n{separator}\n\n"- text_content += rss_content++ # 准备各区域内容映射+ region_contents = {+ "hotlist": stats_content,+ "new_items": new_titles_content,+ "rss": rss_content,+ }++ # 按 region_order 顺序组装内容+ text_content = ""+ for region in region_order:+ content = region_contents.get(region, "")+ if content:+ if text_content:+ text_content += f"\n{separator}\n\n"+ text_content += contentif not text_content:if mode == "incremental":@@ -147,9 +151,10 @@ def render_dingtalk_content(report_data: Dict,update_info: Optional[Dict] = None,mode: str = "daily",- reverse_content_order: bool = False,+ region_order: Optional[List[str]] = None,get_time_func: Optional[Callable[[], datetime]] = None,rss_items: Optional[list] = None,+ show_new_section: bool = True,) -> str:"""渲染钉钉通知内容(支持热榜+RSS合并)@@ -157,13 +162,17 @@ def render_dingtalk_content(report_data: 报告数据字典,包含 stats, new_titles, failed_ids, total_new_countupdate_info: 版本更新信息(可选)mode: 报告模式 ("daily", "incremental", "current")- reverse_content_order: 是否反转内容顺序(新增在前)+ region_order: 区域显示顺序列表get_time_func: 获取当前时间的函数(可选,默认使用 datetime.now())rss_items: RSS 条目列表(可选,用于合并推送)+ show_new_section: 是否显示新增热点区域Returns:格式化的钉钉消息内容"""+ if region_order is None:+ region_order = DEFAULT_REGION_ORDER+total_titles = sum(len(stat["titles"]) for stat in report_data["stats"] if stat["count"] > 0)@@ -209,7 +218,7 @@ def render_dingtalk_content(# 生成新增新闻部分new_titles_content = ""- if report_data["new_titles"]:+ if show_new_section and report_data["new_titles"]:new_titles_content += (f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n")@@ -227,33 +236,30 @@ def render_dingtalk_content(new_titles_content += "\n"- # 根据配置决定内容顺序- text_content = header_content- if reverse_content_order:- # 新增热点在前,热点词汇统计在后- if new_titles_content:- text_content += new_titles_content- if stats_content:- text_content += "\n---\n\n"- if stats_content:- text_content += stats_content- else:- # 默认:热点词汇统计在前,新增热点在后- if stats_content:- text_content += stats_content- if new_titles_content:- text_content += "\n---\n\n"- if new_titles_content:- text_content += new_titles_content-- # 添加 RSS 内容(如果有)+ # RSS 内容+ rss_content = ""if rss_items:rss_content = _render_rss_section_markdown(rss_items)- if stats_content or new_titles_content:- text_content += "\n---\n\n"- text_content += rss_content- if not stats_content and not new_titles_content and not rss_items:+ # 准备各区域内容映射+ region_contents = {+ "hotlist": stats_content,+ "new_items": new_titles_content,+ "rss": rss_content,+ }++ # 按 region_order 顺序组装内容+ text_content = header_content+ has_content = False+ for region in region_order:+ content = region_contents.get(region, "")+ if content:+ if has_content:+ text_content += "\n---\n\n"+ text_content += content+ has_content = True++ if not has_content:if mode == "incremental":mode_text = "增量模式下暂无新增匹配的热点词汇"elif mode == "current":diff --git a/trendradar/notification/senders.py b/trendradar/notification/senders.pyindex 360de7f476..6dc7ac596d 100644--- a/trendradar/notification/senders.py+++ b/trendradar/notification/senders.py@@ -33,7 +33,7 @@ from .batch import add_batch_headers, get_max_batch_header_sizefrom .formatters import convert_markdown_to_mrkdwn, strip_markdown-def _render_ai_analysis(ai_analysis: Any, channel: str, ai_push_mode: str) -> str:+def _render_ai_analysis(ai_analysis: Any, channel: str) -> str:"""渲染 AI 分析内容为指定渠道格式"""if not ai_analysis:return ""@@ -90,7 +90,7 @@ def send_to_feishu(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -126,7 +126,7 @@ def send_to_feishu(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "feishu", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "feishu")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -220,7 +220,7 @@ def send_to_dingtalk(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -255,7 +255,7 @@ def send_to_dingtalk(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "dingtalk", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "dingtalk")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -348,7 +348,7 @@ def send_to_wework(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -395,7 +395,7 @@ def send_to_wework(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "wework", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "wework")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -486,7 +486,7 @@ def send_to_telegram(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -524,7 +524,7 @@ def send_to_telegram(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "telegram", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "telegram")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -606,8 +606,6 @@ def send_to_email(custom_smtp_port: Optional[int] = None,*,get_time_func: Callable = None,- ai_analysis: Any = None,- ai_push_mode: str = "both",) -> bool:"""发送邮件通知@@ -624,6 +622,9 @@ def send_to_email(Returns:bool: 发送是否成功++ Note:+ AI 分析内容已在 HTML 生成时嵌入,无需再追加"""try:if not html_file_path or not Path(html_file_path).exists():@@ -634,12 +635,6 @@ def send_to_email(with open(html_file_path, "r", encoding="utf-8") as f:html_content = f.read()- # 追加 AI 分析内容到 HTML- if ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "email", ai_push_mode)- if ai_content:- html_content = html_content.replace("</body>", f"{ai_content}</body>")-domain = from_email.split("@")[-1].lower()if custom_smtp_server and custom_smtp_port:@@ -776,7 +771,7 @@ def send_to_ntfy(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -838,7 +833,7 @@ def send_to_ntfy(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "ntfy", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "ntfy")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -978,7 +973,7 @@ def send_to_bark(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -1024,7 +1019,7 @@ def send_to_bark(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "bark", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "bark")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -1151,7 +1146,7 @@ def send_to_slack(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -1186,7 +1181,7 @@ def send_to_slack(ai_content = Noneai_stats = Noneif ai_analysis:- ai_content = _render_ai_analysis(ai_analysis, "slack", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "slack")# 提取 AI 分析统计数据(只要 AI 分析成功就显示)if getattr(ai_analysis, "success", False):ai_stats = {@@ -1269,7 +1264,7 @@ def send_to_generic_webhook(rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,ai_analysis: Any = None,- ai_push_mode: str = "both",+ display_regions: Optional[Dict] = None,standalone_data: Optional[Dict] = None,) -> bool:"""@@ -1309,7 +1304,7 @@ def send_to_generic_webhook(ai_stats = Noneif ai_analysis:# 通用 Webhook 使用 markdown 格式渲染 AI 分析- ai_content = _render_ai_analysis(ai_analysis, "wework", ai_push_mode)+ ai_content = _render_ai_analysis(ai_analysis, "wework")# 提取 AI 分析统计数据if getattr(ai_analysis, "success", False):ai_stats = {diff --git a/trendradar/notification/splitter.py b/trendradar/notification/splitter.pyindex f2760e35ef..52f1ee7af7 100644--- a/trendradar/notification/splitter.py+++ b/trendradar/notification/splitter.py@@ -21,6 +21,9 @@ DEFAULT_BATCH_SIZES = {"default": 4000,}+# 默认区域顺序+DEFAULT_REGION_ORDER = ["hotlist", "rss", "new_items", "standalone", "ai_analysis"]+def split_content_into_batches(report_data: Dict,@@ -30,7 +33,7 @@ def split_content_into_batches(mode: str = "daily",batch_sizes: Optional[Dict[str, int]] = None,feishu_separator: str = "---",- reverse_content_order: bool = False,+ region_order: Optional[List[str]] = None,get_time_func: Optional[Callable[[], datetime]] = None,rss_items: Optional[list] = None,rss_new_items: Optional[list] = None,@@ -41,13 +44,14 @@ def split_content_into_batches(rank_threshold: int = 10,ai_stats: Optional[Dict] = None,report_type: str = "热点分析报告",+ show_new_section: bool = True,) -> List[str]:"""分批处理消息内容,确保词组标题+至少第一条新闻的完整性(支持热榜+RSS合并+AI分析+独立展示区)热榜统计与RSS统计并列显示,热榜新增与RSS新增并列显示。- reverse_content_order 控制统计和新增的前后顺序。- AI分析内容默认放在最后(footer之前)。- 独立展示区放在新增区块之后、失败ID之前。+ region_order 控制各区域的显示顺序。+ AI分析内容根据 region_order 中的位置显示。+ 独立展示区根据 region_order 中的位置显示。Args:report_data: 报告数据字典,包含 stats, new_titles, failed_ids, total_new_count@@ -57,7 +61,7 @@ def split_content_into_batches(mode: 报告模式 (daily, incremental, current)batch_sizes: 批次大小配置字典(可选)feishu_separator: 飞书消息分隔符- reverse_content_order: 是否反转内容顺序(新增在前,统计在后)+ region_order: 区域显示顺序列表get_time_func: 获取当前时间的函数(可选)rss_items: RSS 统计条目列表(按源分组,用于合并推送)rss_new_items: RSS 新增条目列表(可选,用于新增区块)@@ -70,6 +74,8 @@ def split_content_into_batches(Returns:分批后的消息内容列表"""+ if region_order is None:+ region_order = DEFAULT_REGION_ORDER# 合并批次大小配置sizes = {**DEFAULT_BATCH_SIZES, **(batch_sizes or {})}@@ -212,15 +218,31 @@ def split_content_into_batches(return batches# 定义处理热点词汇统计的函数- def process_stats_section(current_batch, current_batch_has_content, batches):+ def process_stats_section(current_batch, current_batch_has_content, batches, add_separator=True):"""处理热点词汇统计"""if not report_data["stats"]:return current_batch, current_batch_has_content, batchestotal_count = len(report_data["stats"])+ # 根据 add_separator 决定是否添加前置分割线+ actual_stats_header = ""+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type == "feishu":+ actual_stats_header = f"\n{feishu_separator}\n\n{stats_header}"+ elif format_type == "dingtalk":+ actual_stats_header = f"\n---\n\n{stats_header}"+ elif format_type in ("wework", "bark"):+ actual_stats_header = f"\n\n\n\n{stats_header}"+ else:+ actual_stats_header = f"\n\n{stats_header}"+ else:+ # 不需要分割线(第一个区域)+ actual_stats_header = stats_header+# 添加统计标题- test_content = current_batch + stats_header+ test_content = current_batch + actual_stats_headerif (len(test_content.encode("utf-8")) + len(base_footer.encode("utf-8"))< max_bytes@@ -230,6 +252,7 @@ def split_content_into_batches(else:if current_batch_has_content:batches.append(current_batch + base_footer)+ # 新批次开头不需要分割线,使用原始 stats_headercurrent_batch = base_header + stats_headercurrent_batch_has_content = True@@ -430,26 +453,43 @@ def split_content_into_batches(return current_batch, current_batch_has_content, batches# 定义处理新增新闻的函数- def process_new_titles_section(current_batch, current_batch_has_content, batches):+ def process_new_titles_section(current_batch, current_batch_has_content, batches, add_separator=True):"""处理新增新闻"""- if not report_data["new_titles"]:+ if not show_new_section or not report_data["new_titles"]:return current_batch, current_batch_has_content, batches+ # 根据 add_separator 决定是否添加前置分割线new_header = ""- if format_type in ("wework", "bark"):- new_header = f"\n\n\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"- elif format_type == "telegram":- new_header = (- f"\n\n🆕 本次新增热点新闻 (共 {report_data['total_new_count']} 条)\n\n"- )- elif format_type == "ntfy":- new_header = f"\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"- elif format_type == "feishu":- new_header = f"\n{feishu_separator}\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"- elif format_type == "dingtalk":- new_header = f"\n---\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"- elif format_type == "slack":- new_header = f"\n\n🆕 *本次新增热点新闻* (共 {report_data['total_new_count']} 条)\n\n"+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type in ("wework", "bark"):+ new_header = f"\n\n\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "telegram":+ new_header = (+ f"\n\n🆕 本次新增热点新闻 (共 {report_data['total_new_count']} 条)\n\n"+ )+ elif format_type == "ntfy":+ new_header = f"\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "feishu":+ new_header = f"\n{feishu_separator}\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "dingtalk":+ new_header = f"\n---\n\n🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "slack":+ new_header = f"\n\n🆕 *本次新增热点新闻* (共 {report_data['total_new_count']} 条)\n\n"+ else:+ # 不需要分割线(第一个区域)+ if format_type in ("wework", "bark"):+ new_header = f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "telegram":+ new_header = f"🆕 本次新增热点新闻 (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "ntfy":+ new_header = f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "feishu":+ new_header = f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "dingtalk":+ new_header = f"🆕 **本次新增热点新闻** (共 {report_data['total_new_count']} 条)\n\n"+ elif format_type == "slack":+ new_header = f"🆕 *本次新增热点新闻* (共 {report_data['total_new_count']} 条)\n\n"test_content = current_batch + new_headerif (@@ -578,59 +618,137 @@ def split_content_into_batches(return current_batch, current_batch_has_content, batches- # 根据配置决定处理顺序- if reverse_content_order:- # 新增热点在前,热点词汇统计在后- # 1. 处理热榜新增- current_batch, current_batch_has_content, batches = process_new_titles_section(- current_batch, current_batch_has_content, batches+ # 定义处理 AI 分析的函数+ def process_ai_section(current_batch, current_batch_has_content, batches, add_separator=True):+ """处理 AI 分析内容"""+ nonlocal ai_content+ if not ai_content:+ return current_batch, current_batch_has_content, batches++ # 根据 add_separator 决定是否添加前置分割线+ ai_separator = ""+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type == "feishu":+ ai_separator = f"\n{feishu_separator}\n\n"+ elif format_type == "dingtalk":+ ai_separator = "\n---\n\n"+ elif format_type in ("wework", "bark"):+ ai_separator = "\n\n\n\n"+ elif format_type in ("telegram", "ntfy", "slack"):+ ai_separator = "\n\n"+ # 如果不需要分割线,ai_separator 保持为空字符串++ # 尝试将 AI 内容添加到当前批次+ test_content = current_batch + ai_separator + ai_content+ if (+ len(test_content.encode("utf-8")) + len(base_footer.encode("utf-8"))+ < max_bytes+ ):+ current_batch = test_content+ current_batch_has_content = True+ else:+ # 当前批次容纳不下,开启新批次+ if current_batch_has_content:+ batches.append(current_batch + base_footer)+ # AI 内容可能很长,需要考虑是否需要进一步分割+ ai_with_header = base_header + ai_content+ current_batch = ai_with_header+ current_batch_has_content = True++ return current_batch, current_batch_has_content, batches++ # 定义处理独立展示区的函数+ def process_standalone_section_wrapper(current_batch, current_batch_has_content, batches, add_separator=True):+ """处理独立展示区"""+ if not standalone_data:+ return current_batch, current_batch_has_content, batches+ return _process_standalone_section(+ standalone_data, format_type, feishu_separator, base_header, base_footer,+ max_bytes, current_batch, current_batch_has_content, batches, timezone,+ rank_threshold, add_separator)- # 2. 处理 RSS 新增(如果有)- if rss_new_items:- current_batch, current_batch_has_content, batches = _process_rss_new_titles_section(- rss_new_items, format_type, feishu_separator, base_header, base_footer,- max_bytes, current_batch, current_batch_has_content, batches, timezone- )- # 3. 处理热榜统计- current_batch, current_batch_has_content, batches = process_stats_section(- current_batch, current_batch_has_content, batches++ # 定义处理 RSS 统计的函数+ def process_rss_stats_wrapper(current_batch, current_batch_has_content, batches, add_separator=True):+ """处理 RSS 统计"""+ if not rss_items:+ return current_batch, current_batch_has_content, batches+ return _process_rss_stats_section(+ rss_items, format_type, feishu_separator, base_header, base_footer,+ max_bytes, current_batch, current_batch_has_content, batches, timezone,+ add_separator)- # 4. 处理 RSS 统计(如果有)- if rss_items:- current_batch, current_batch_has_content, batches = _process_rss_stats_section(- rss_items, format_type, feishu_separator, base_header, base_footer,- max_bytes, current_batch, current_batch_has_content, batches, timezone- )- else:- # 默认:热点词汇统计在前,新增热点在后- # 1. 处理热榜统计- current_batch, current_batch_has_content, batches = process_stats_section(- current_batch, current_batch_has_content, batches++ # 定义处理 RSS 新增的函数+ def process_rss_new_wrapper(current_batch, current_batch_has_content, batches, add_separator=True):+ """处理 RSS 新增"""+ if not rss_new_items:+ return current_batch, current_batch_has_content, batches+ return _process_rss_new_titles_section(+ rss_new_items, format_type, feishu_separator, base_header, base_footer,+ max_bytes, current_batch, current_batch_has_content, batches, timezone,+ add_separator)- # 2. 处理 RSS 统计(如果有)- if rss_items:- current_batch, current_batch_has_content, batches = _process_rss_stats_section(- rss_items, format_type, feishu_separator, base_header, base_footer,- max_bytes, current_batch, current_batch_has_content, batches, timezone++ # 按 region_order 顺序处理各区域+ # 记录是否已有区域内容(用于决定是否添加分割线)+ has_region_content = False++ for region in region_order:+ # 记录处理前的状态,用于判断该区域是否产生了内容+ batch_before = current_batch+ has_content_before = current_batch_has_content+ batches_len_before = len(batches)++ # 决定是否需要添加分割线(第一个有内容的区域不需要)+ add_separator = has_region_content++ if region == "hotlist":+ # 处理热榜统计+ current_batch, current_batch_has_content, batches = process_stats_section(+ current_batch, current_batch_has_content, batches, add_separator)- # 3. 处理热榜新增- current_batch, current_batch_has_content, batches = process_new_titles_section(- current_batch, current_batch_has_content, batches- )- # 4. 处理 RSS 新增(如果有)- if rss_new_items:- current_batch, current_batch_has_content, batches = _process_rss_new_titles_section(- rss_new_items, format_type, feishu_separator, base_header, base_footer,- max_bytes, current_batch, current_batch_has_content, batches, timezone+ elif region == "rss":+ # 处理 RSS 统计+ current_batch, current_batch_has_content, batches = process_rss_stats_wrapper(+ current_batch, current_batch_has_content, batches, add_separator+ )+ elif region == "new_items":+ # 处理热榜新增+ current_batch, current_batch_has_content, batches = process_new_titles_section(+ current_batch, current_batch_has_content, batches, add_separator+ )+ # 处理 RSS 新增(跟随 new_items,继承 add_separator 逻辑)+ # 如果热榜新增产生了内容,RSS 新增需要分割线+ new_batch_changed = (+ current_batch != batch_before or+ current_batch_has_content != has_content_before or+ len(batches) != batches_len_before+ )+ rss_new_separator = new_batch_changed or has_region_content+ current_batch, current_batch_has_content, batches = process_rss_new_wrapper(+ current_batch, current_batch_has_content, batches, rss_new_separator+ )+ elif region == "standalone":+ # 处理独立展示区+ current_batch, current_batch_has_content, batches = process_standalone_section_wrapper(+ current_batch, current_batch_has_content, batches, add_separator+ )+ elif region == "ai_analysis":+ # 处理 AI 分析+ current_batch, current_batch_has_content, batches = process_ai_section(+ current_batch, current_batch_has_content, batches, add_separator)- # 5. 处理独立展示区(如果有)- if standalone_data:- current_batch, current_batch_has_content, batches = _process_standalone_section(- standalone_data, format_type, feishu_separator, base_header, base_footer,- max_bytes, current_batch, current_batch_has_content, batches, timezone,- rank_threshold+ # 检查该区域是否产生了内容+ region_produced_content = (+ current_batch != batch_before or+ current_batch_has_content != has_content_before or+ len(batches) != batches_len_before)+ if region_produced_content:+ has_region_content = Trueif report_data["failed_ids"]:failed_header = ""@@ -679,41 +797,6 @@ def split_content_into_batches(current_batch = test_contentcurrent_batch_has_content = True- # 处理 AI 分析内容(放在最后,footer 之前)- if ai_content:- # 添加 AI 分析区块分隔符- ai_separator = ""- if format_type == "feishu":- ai_separator = f"\n{feishu_separator}\n\n"- elif format_type == "dingtalk":- ai_separator = "\n---\n\n"- elif format_type in ("wework", "bark"):- ai_separator = "\n\n\n\n"- elif format_type in ("telegram", "ntfy", "slack"):- ai_separator = "\n\n"-- # 尝试将 AI 内容添加到当前批次- test_content = current_batch + ai_separator + ai_content- if (- len(test_content.encode("utf-8")) + len(base_footer.encode("utf-8"))- < max_bytes- ):- current_batch = test_content- current_batch_has_content = True- else:- # 当前批次容纳不下,开启新批次- if current_batch_has_content:- batches.append(current_batch + base_footer)- # AI 内容可能很长,需要考虑是否需要进一步分割- ai_with_header = base_header + ai_content- if len(ai_with_header.encode("utf-8")) + len(base_footer.encode("utf-8")) < max_bytes:- current_batch = ai_with_header- current_batch_has_content = True- else:- # AI 内容过长,直接添加(可能会超限,但保持完整性)- current_batch = ai_with_header- current_batch_has_content = True-# 完成最后批次if current_batch_has_content:batches.append(current_batch + base_footer)@@ -732,6 +815,7 @@ def _process_rss_stats_section(current_batch_has_content: bool,batches: List[str],timezone: str = "Asia/Shanghai",+ add_separator: bool = True,) -> tuple:"""处理 RSS 统计区块(按关键词分组,与热榜统计格式一致)@@ -747,6 +831,7 @@ def _process_rss_stats_section(current_batch_has_content: 当前批次是否有内容batches: 已完成的批次列表timezone: 时区名称+ add_separator: 是否在区块前添加分割线(第一个区域时为 False)Returns:(current_batch, current_batch_has_content, batches) 元组@@ -758,18 +843,34 @@ def _process_rss_stats_section(total_items = sum(stat["count"] for stat in rss_stats)total_keywords = len(rss_stats)- # RSS 统计区块标题+ # RSS 统计区块标题(根据 add_separator 决定是否添加前置分割线)rss_header = ""- if format_type == "feishu":- rss_header = f"\n{feishu_separator}\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"- elif format_type == "dingtalk":- rss_header = f"\n---\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"- elif format_type == "telegram":- rss_header = f"\n\n📰 RSS 订阅统计 (共 {total_items} 条)\n\n"- elif format_type == "slack":- rss_header = f"\n\n📰 *RSS 订阅统计* (共 {total_items} 条)\n\n"+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type == "feishu":+ rss_header = f"\n{feishu_separator}\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ rss_header = f"\n---\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ elif format_type in ("wework", "bark"):+ rss_header = f"\n\n\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ rss_header = f"\n\n📰 RSS 订阅统计 (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ rss_header = f"\n\n📰 *RSS 订阅统计* (共 {total_items} 条)\n\n"+ else:+ rss_header = f"\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"else:- rss_header = f"\n\n📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ # 不需要分割线(第一个区域)+ if format_type == "feishu":+ rss_header = f"📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ rss_header = f"📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ rss_header = f"📰 RSS 订阅统计 (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ rss_header = f"📰 *RSS 订阅统计* (共 {total_items} 条)\n\n"+ else:+ rss_header = f"📰 **RSS 订阅统计** (共 {total_items} 条)\n\n"# 添加 RSS 标题test_content = current_batch + rss_header@@ -937,6 +1038,7 @@ def _process_rss_new_titles_section(current_batch_has_content: bool,batches: List[str],timezone: str = "Asia/Shanghai",+ add_separator: bool = True,) -> tuple:"""处理 RSS 新增区块(按来源分组,与热榜新增格式一致)@@ -952,6 +1054,7 @@ def _process_rss_new_titles_section(current_batch_has_content: 当前批次是否有内容batches: 已完成的批次列表timezone: 时区名称+ add_separator: 是否在区块前添加分割线(第一个区域时为 False)Returns:(current_batch, current_batch_has_content, batches) 元组@@ -974,20 +1077,36 @@ def _process_rss_new_titles_section(# 计算总条目数total_items = sum(len(titles) for titles in source_map.values())- # RSS 新增区块标题+ # RSS 新增区块标题(根据 add_separator 决定是否添加前置分割线)new_header = ""- if format_type in ("wework", "bark"):- new_header = f"\n\n\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"- elif format_type == "telegram":- new_header = f"\n\n🆕 RSS 本次新增 (共 {total_items} 条)\n\n"- elif format_type == "ntfy":- new_header = f"\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"- elif format_type == "feishu":- new_header = f"\n{feishu_separator}\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"- elif format_type == "dingtalk":- new_header = f"\n---\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"- elif format_type == "slack":- new_header = f"\n\n🆕 *RSS 本次新增* (共 {total_items} 条)\n\n"+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type in ("wework", "bark"):+ new_header = f"\n\n\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ new_header = f"\n\n🆕 RSS 本次新增 (共 {total_items} 条)\n\n"+ elif format_type == "ntfy":+ new_header = f"\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "feishu":+ new_header = f"\n{feishu_separator}\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ new_header = f"\n---\n\n🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ new_header = f"\n\n🆕 *RSS 本次新增* (共 {total_items} 条)\n\n"+ else:+ # 不需要分割线(第一个区域)+ if format_type in ("wework", "bark"):+ new_header = f"🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ new_header = f"🆕 RSS 本次新增 (共 {total_items} 条)\n\n"+ elif format_type == "ntfy":+ new_header = f"🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "feishu":+ new_header = f"🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ new_header = f"🆕 **RSS 本次新增** (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ new_header = f"🆕 *RSS 本次新增* (共 {total_items} 条)\n\n"# 添加 RSS 新增标题test_content = current_batch + new_header@@ -1160,6 +1279,7 @@ def _process_standalone_section(batches: List[str],timezone: str = "Asia/Shanghai",rank_threshold: int = 10,+ add_separator: bool = True,) -> tuple:"""处理独立展示区区块@@ -1181,6 +1301,8 @@ def _process_standalone_section(current_batch_has_content: 当前批次是否有内容batches: 已完成的批次列表timezone: 时区名称+ rank_threshold: 排名高亮阈值+ add_separator: 是否在区块前添加分割线(第一个区域时为 False)Returns:(current_batch, current_batch_has_content, batches) 元组@@ -1199,18 +1321,34 @@ def _process_standalone_section(total_rss_items = sum(len(f.get("items", [])) for f in rss_feeds)total_items = total_platform_items + total_rss_items- # 独立展示区标题+ # 独立展示区标题(根据 add_separator 决定是否添加前置分割线)section_header = ""- if format_type == "feishu":- section_header = f"\n{feishu_separator}\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"- elif format_type == "dingtalk":- section_header = f"\n---\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"- elif format_type == "telegram":- section_header = f"\n\n📋 独立展示区 (共 {total_items} 条)\n\n"- elif format_type == "slack":- section_header = f"\n\n📋 *独立展示区* (共 {total_items} 条)\n\n"+ if add_separator and current_batch_has_content:+ # 需要添加分割线+ if format_type == "feishu":+ section_header = f"\n{feishu_separator}\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ section_header = f"\n---\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"+ elif format_type in ("wework", "bark"):+ section_header = f"\n\n\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ section_header = f"\n\n📋 独立展示区 (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ section_header = f"\n\n📋 *独立展示区* (共 {total_items} 条)\n\n"+ else:+ section_header = f"\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"else:- section_header = f"\n\n📋 **独立展示区** (共 {total_items} 条)\n\n"+ # 不需要分割线(第一个区域)+ if format_type == "feishu":+ section_header = f"📋 **独立展示区** (共 {total_items} 条)\n\n"+ elif format_type == "dingtalk":+ section_header = f"📋 **独立展示区** (共 {total_items} 条)\n\n"+ elif format_type == "telegram":+ section_header = f"📋 独立展示区 (共 {total_items} 条)\n\n"+ elif format_type == "slack":+ section_header = f"📋 *独立展示区* (共 {total_items} 条)\n\n"+ else:+ section_header = f"📋 **独立展示区** (共 {total_items} 条)\n\n"# 添加区块标题test_content = current_batch + section_headerdiff --git a/trendradar/report/generator.py b/trendradar/report/generator.pyindex f62f417e29..374ea0b7d4 100644--- a/trendradar/report/generator.py+++ b/trendradar/report/generator.py@@ -20,6 +20,7 @@ def prepare_report_data(rank_threshold: int = 3,matches_word_groups_func: Optional[Callable] = None,load_frequency_words_func: Optional[Callable] = None,+ show_new_section: bool = True,) -> Dict:"""准备报告数据@@ -33,14 +34,15 @@ def prepare_report_data(rank_threshold: 排名阈值matches_word_groups_func: 词组匹配函数load_frequency_words_func: 加载频率词函数+ show_new_section: 是否显示新增热点区域Returns:Dict: 准备好的报告数据"""processed_new_titles = []- # 在增量模式下隐藏新增新闻区域- hide_new_section = mode == "incremental"+ # 在增量模式下或配置关闭时隐藏新增新闻区域+ hide_new_section = mode == "incremental" or not show_new_section# 只有在非隐藏模式下才处理新增新闻部分if not hide_new_section:@@ -144,7 +146,6 @@ def generate_html_report(new_titles: Optional[Dict] = None,id_to_name: Optional[Dict] = None,mode: str = "daily",- is_daily_summary: bool = False,update_info: Optional[Dict] = None,rank_threshold: int = 3,output_dir: str = "output",@@ -153,11 +154,15 @@ def generate_html_report(render_html_func: Optional[Callable] = None,matches_word_groups_func: Optional[Callable] = None,load_frequency_words_func: Optional[Callable] = None,- enable_index_copy: bool = True,) -> str:"""生成 HTML 报告+ 每次生成 HTML 后会:+ 1. 保存时间戳快照到 output/html/日期/时间.html(历史记录)+ 2. 复制到 output/html/latest/{mode}.html(最新报告)+ 3. 复制到 output/index.html 和根目录 index.html(入口)+Args:stats: 统计结果列表total_titles: 总标题数@@ -165,7 +170,6 @@ def generate_html_report(new_titles: 新增标题id_to_name: ID 到名称的映射mode: 报告模式 (daily/incremental/current)- is_daily_summary: 是否是每日汇总update_info: 更新信息rank_threshold: 排名阈值output_dir: 输出目录@@ -174,25 +178,17 @@ def generate_html_report(render_html_func: HTML 渲染函数matches_word_groups_func: 词组匹配函数load_frequency_words_func: 加载频率词函数- enable_index_copy: 是否复制到 index.htmlReturns:- str: 生成的 HTML 文件路径+ str: 生成的 HTML 文件路径(时间戳快照路径)"""- if is_daily_summary:- if mode == "current":- filename = "当前榜单汇总.html"- elif mode == "incremental":- filename = "当日增量.html"- else:- filename = "当日汇总.html"- else:- filename = f"{time_filename}.html"+ # 时间戳快照文件名+ snapshot_filename = f"{time_filename}.html"- # 构建输出路径- output_path = Path(output_dir) / date_folder / "html"- output_path.mkdir(parents=True, exist_ok=True)- file_path = str(output_path / filename)+ # 构建输出路径(扁平化结构:output/html/日期/)+ snapshot_path = Path(output_dir) / "html" / date_folder+ snapshot_path.mkdir(parents=True, exist_ok=True)+ snapshot_file = str(snapshot_path / snapshot_filename)# 准备报告数据report_data = prepare_report_data(@@ -209,27 +205,32 @@ def generate_html_report(# 渲染 HTML 内容if render_html_func:html_content = render_html_func(- report_data, total_titles, is_daily_summary, mode, update_info+ report_data, total_titles, mode, update_info)else:# 默认简单 HTMLhtml_content = f"<html><body><h1>Report</h1><pre>{report_data}</pre></body></html>"- # 写入文件- with open(file_path, "w", encoding="utf-8") as f:+ # 1. 保存时间戳快照(历史记录)+ with open(snapshot_file, "w", encoding="utf-8") as f:+ f.write(html_content)++ # 2. 复制到 html/latest/{mode}.html(最新报告)+ latest_dir = Path(output_dir) / "html" / "latest"+ latest_dir.mkdir(parents=True, exist_ok=True)+ latest_file = latest_dir / f"{mode}.html"+ with open(latest_file, "w", encoding="utf-8") as f:f.write(html_content)- # 如果是每日汇总且启用 index 复制- if is_daily_summary and enable_index_copy:- # 生成到根目录(供 GitHub Pages 访问)- root_index_path = Path("index.html")- with open(root_index_path, "w", encoding="utf-8") as f:- f.write(html_content)+ # 3. 复制到 index.html(入口)+ # output/index.html(供 Docker Volume 挂载访问)+ output_index = Path(output_dir) / "index.html"+ with open(output_index, "w", encoding="utf-8") as f:+ f.write(html_content)- # 同时生成到 output 目录(供 Docker Volume 挂载访问)- output_index_path = Path(output_dir) / "index.html"- Path(output_dir).mkdir(parents=True, exist_ok=True)- with open(output_index_path, "w", encoding="utf-8") as f:- f.write(html_content)+ # 根目录 index.html(供 GitHub Pages 访问)+ root_index = Path("index.html")+ with open(root_index, "w", encoding="utf-8") as f:+ f.write(html_content)- return file_path+ return snapshot_filediff --git a/trendradar/report/html.py b/trendradar/report/html.pyindex 020e291bcc..9608568927 100644--- a/trendradar/report/html.py+++ b/trendradar/report/html.py@@ -6,44 +6,52 @@ HTML 报告渲染模块"""from datetime import datetime-from typing import Dict, List, Optional, Callable+from typing import Any, Dict, List, Optional, Callablefrom trendradar.report.helpers import html_escapefrom trendradar.utils.time import convert_time_for_display+from trendradar.ai.formatter import render_ai_analysis_html_richdef render_html_content(report_data: Dict,total_titles: int,- is_daily_summary: bool = False,mode: str = "daily",update_info: Optional[Dict] = None,*,- reverse_content_order: bool = False,+ region_order: Optional[List[str]] = None,get_time_func: Optional[Callable[[], datetime]] = None,rss_items: Optional[List[Dict]] = None,rss_new_items: Optional[List[Dict]] = None,display_mode: str = "keyword",standalone_data: Optional[Dict] = None,+ ai_analysis: Optional[Any] = None,+ show_new_section: bool = True,) -> str:"""渲染HTML内容Args:report_data: 报告数据字典,包含 stats, new_titles, failed_ids, total_new_counttotal_titles: 新闻总数- is_daily_summary: 是否为当日汇总mode: 报告模式 ("daily", "current", "incremental")update_info: 更新信息(可选)- reverse_content_order: 是否反转内容顺序(新增热点在前)+ region_order: 区域显示顺序列表get_time_func: 获取当前时间的函数(可选,默认使用 datetime.now)rss_items: RSS 统计条目列表(可选)rss_new_items: RSS 新增条目列表(可选)display_mode: 显示模式 ("keyword"=按关键词分组, "platform"=按平台分组)standalone_data: 独立展示区数据(可选),包含 platforms 和 rss_feeds+ ai_analysis: AI 分析结果对象(可选),AIAnalysisResult 实例+ show_new_section: 是否显示新增热点区域Returns:渲染后的 HTML 字符串"""+ # 默认区域顺序+ default_region_order = ["hotlist", "rss", "new_items", "standalone", "ai_analysis"]+ if region_order is None:+ region_order = default_region_order+html = """<!DOCTYPE html><html>@@ -318,10 +326,21 @@ def render_html_content(color: #7c3aed;}+ /* 通用区域分割线样式 */+ .section-divider {+ margin-top: 32px;+ padding-top: 24px;+ border-top: 2px solid #e5e7eb;+ }++ /* 热榜统计区样式 */+ .hotlist-section {+ /* 默认无边框,由 section-divider 动态添加 */+ }+.new-section {margin-top: 40px;padding-top: 24px;- border-top: 2px solid #f0f0f0;}.new-section-title {@@ -487,7 +506,6 @@ def render_html_content(.rss-section {margin-top: 32px;padding-top: 24px;- border-top: 2px solid #e5e7eb;}.rss-section-header {@@ -600,7 +618,6 @@ def render_html_content(.standalone-section {margin-top: 32px;padding-top: 24px;- border-top: 2px solid #e5e7eb;}.standalone-section-header {@@ -613,7 +630,7 @@ def render_html_content(.standalone-section-title {font-size: 18px;font-weight: 600;- color: #4f46e5;+ color: #059669;}.standalone-section-count {@@ -649,6 +666,72 @@ def render_html_content(font-size: 13px;font-weight: 500;}++ /* AI 分析区块样式 */+ .ai-section {+ margin-top: 32px;+ padding: 24px;+ background: linear-gradient(135deg, #f0f9ff 0%, #e0f2fe 100%);+ border-radius: 12px;+ border: 1px solid #bae6fd;+ }++ .ai-section-header {+ display: flex;+ align-items: center;+ gap: 10px;+ margin-bottom: 20px;+ }++ .ai-section-title {+ font-size: 18px;+ font-weight: 600;+ color: #0369a1;+ }++ .ai-section-badge {+ background: #0ea5e9;+ color: white;+ font-size: 11px;+ font-weight: 600;+ padding: 3px 8px;+ border-radius: 4px;+ }++ .ai-block {+ margin-bottom: 16px;+ padding: 16px;+ background: white;+ border-radius: 8px;+ box-shadow: 0 1px 3px rgba(0,0,0,0.05);+ }++ .ai-block:last-child {+ margin-bottom: 0;+ }++ .ai-block-title {+ font-size: 14px;+ font-weight: 600;+ color: #0369a1;+ margin-bottom: 8px;+ }++ .ai-block-content {+ font-size: 14px;+ line-height: 1.6;+ color: #334155;+ white-space: pre-wrap;+ }++ .ai-error {+ padding: 16px;+ background: #fef2f2;+ border: 1px solid #fecaca;+ border-radius: 8px;+ color: #991b1b;+ font-size: 14px;+ }</style></head><body>@@ -664,16 +747,13 @@ def render_html_content(<span class="info-label">报告类型</span><span class="info-value">"""- # 处理报告类型显示- if is_daily_summary:- if mode == "current":- html += "当前榜单"- elif mode == "incremental":- html += "增量模式"- else:- html += "当日汇总"+ # 处理报告类型显示(根据 mode 直接显示)+ if mode == "current":+ html += "当前榜单"+ elif mode == "incremental":+ html += "增量分析"else:- html += "实时分析"+ html += "全天汇总"html += """</span></div>@@ -837,9 +917,15 @@ def render_html_content(stats_html += """</div>"""+ # 给热榜统计添加外层包装+ if stats_html:+ stats_html = f"""+ <div class="hotlist-section">{stats_html}+ </div>"""+# 生成新增新闻区域的HTMLnew_titles_html = ""- if report_data["new_titles"]:+ if show_new_section and report_data["new_titles"]:new_titles_html += f"""<div class="new-section"><div class="new-section-title">本次新增热点 (共 {report_data['total_new_count']} 条)</div>"""@@ -1062,7 +1148,7 @@ def render_html_content(standalone_html = f"""<div class="standalone-section"><div class="standalone-section-header">- <div class="standalone-section-title">📋 独立展示区</div>+ <div class="standalone-section-title">独立展示区</div><div class="standalone-section-count">{total_count} 条</div></div>"""@@ -1231,15 +1317,50 @@ def render_html_content(# 生成独立展示区 HTMLstandalone_html = render_standalone_html(standalone_data)- # 根据配置决定内容顺序(与推送逻辑一致)- if reverse_content_order:- # 新增在前,统计在后- # 顺序:热榜新增 → RSS新增 → 热榜统计 → RSS统计 → 独立展示区- html += new_titles_html + rss_new_html + stats_html + rss_stats_html + standalone_html- else:- # 默认:统计在前,新增在后- # 顺序:热榜统计 → RSS统计 → 热榜新增 → RSS新增 → 独立展示区- html += stats_html + rss_stats_html + new_titles_html + rss_new_html + standalone_html+ # 生成 AI 分析 HTML+ ai_html = render_ai_analysis_html_rich(ai_analysis) if ai_analysis else ""++ # 准备各区域内容映射+ region_contents = {+ "hotlist": stats_html,+ "rss": rss_stats_html,+ "new_items": (new_titles_html, rss_new_html), # 元组,分别处理+ "standalone": standalone_html,+ "ai_analysis": ai_html,+ }++ def add_section_divider(content: str) -> str:+ """为内容的外层 div 添加 section-divider 类"""+ if not content or 'class="' not in content:+ return content+ first_class_pos = content.find('class="')+ if first_class_pos != -1:+ insert_pos = first_class_pos + len('class="')+ return content[:insert_pos] + "section-divider " + content[insert_pos:]+ return content++ # 按 region_order 顺序组装内容,动态添加分割线+ has_previous_content = False+ for region in region_order:+ content = region_contents.get(region, "")+ if region == "new_items":+ # 特殊处理 new_items 区域(包含热榜新增和 RSS 新增两部分)+ new_html, rss_new = content+ if new_html:+ if has_previous_content:+ new_html = add_section_divider(new_html)+ html += new_html+ has_previous_content = True+ if rss_new:+ if has_previous_content:+ rss_new = add_section_divider(rss_new)+ html += rss_new+ has_previous_content = True+ elif content:+ if has_previous_content:+ content = add_section_divider(content)+ html += content+ has_previous_content = Truehtml += """</div>diff --git a/trendradar/storage/__init__.py b/trendradar/storage/__init__.pyindex 1897feb54b..689be14c62 100644--- a/trendradar/storage/__init__.py+++ b/trendradar/storage/__init__.py@@ -12,9 +12,12 @@ from trendradar.storage.base import (StorageBackend,NewsItem,NewsData,+ RSSItem,+ RSSData,convert_crawl_results_to_news_data,convert_news_data_to_results,)+from trendradar.storage.sqlite_mixin import SQLiteStorageMixinfrom trendradar.storage.local import LocalStorageBackendfrom trendradar.storage.manager import StorageManager, get_storage_manager@@ -31,6 +34,10 @@ __all__ = ["StorageBackend","NewsItem","NewsData",+ "RSSItem",+ "RSSData",+ # Mixin+ "SQLiteStorageMixin",# 转换函数"convert_crawl_results_to_news_data","convert_news_data_to_results",diff --git a/trendradar/storage/base.py b/trendradar/storage/base.pyindex 5f06645f5a..652937f84b 100644--- a/trendradar/storage/base.py+++ b/trendradar/storage/base.py@@ -27,6 +27,9 @@ class NewsItem:first_time: str = "" # 首次出现时间last_time: str = "" # 最后出现时间count: int = 1 # 出现次数+ rank_timeline: List[Dict[str, Any]] = field(default_factory=list) # 完整排名时间线+ # 格式: [{"time": "09:30", "rank": 1}, {"time": "10:00", "rank": 2}, ...]+ # None 表示脱榜: [{"time": "11:00", "rank": None}]def to_dict(self) -> Dict[str, Any]:"""转换为字典"""@@ -42,6 +45,7 @@ class NewsItem:"first_time": self.first_time,"last_time": self.last_time,"count": self.count,+ "rank_timeline": self.rank_timeline,}@classmethod@@ -59,6 +63,7 @@ class NewsItem:first_time=data.get("first_time", ""),last_time=data.get("last_time", ""),count=data.get("count", 1),+ rank_timeline=data.get("rank_timeline", []),)diff --git a/trendradar/storage/local.py b/trendradar/storage/local.pyindex 9e197e983b..3dbc5d4dac 100644--- a/trendradar/storage/local.py+++ b/trendradar/storage/local.py@@ -14,15 +14,15 @@ from pathlib import Pathfrom typing import Dict, List, Optionalfrom trendradar.storage.base import StorageBackend, NewsItem, NewsData, RSSItem, RSSData+from trendradar.storage.sqlite_mixin import SQLiteStorageMixinfrom trendradar.utils.time import (get_configured_time,format_date_folder,format_time_filename,)-from trendradar.utils.url import normalize_url-class LocalStorageBackend(StorageBackend):+class LocalStorageBackend(SQLiteStorageMixin, StorageBackend):"""本地存储后端@@ -62,6 +62,10 @@ class LocalStorageBackend(StorageBackend):def supports_txt(self) -> bool:return self.enable_txt+ # ========================================+ # SQLiteStorageMixin 抽象方法实现+ # ========================================+def _get_configured_time(self) -> datetime:"""获取配置时区的当前时间"""return get_configured_time(self.timezone)@@ -115,510 +119,112 @@ class LocalStorageBackend(StorageBackend):return self._db_connections[db_path]- def _get_schema_path(self, db_type: str = "news") -> Path:- """- 获取 schema.sql 文件路径-- Args:- db_type: 数据库类型 ("news" 或 "rss")-- Returns:- schema 文件路径- """- if db_type == "rss":- return Path(__file__).parent / "rss_schema.sql"- return Path(__file__).parent / "schema.sql"-- def _init_tables(self, conn: sqlite3.Connection, db_type: str = "news") -> None:- """- 从 schema.sql 初始化数据库表结构-- Args:- conn: 数据库连接- db_type: 数据库类型 ("news" 或 "rss")- """- schema_path = self._get_schema_path(db_type)-- if schema_path.exists():- with open(schema_path, "r", encoding="utf-8") as f:- schema_sql = f.read()- conn.executescript(schema_sql)- else:- raise FileNotFoundError(f"Schema file not found: {schema_path}")-- conn.commit()+ # ========================================+ # StorageBackend 接口实现(委托给 mixin)+ # ========================================def save_news_data(self, data: NewsData) -> bool:- """- 保存新闻数据到 SQLite(以 URL 为唯一标识,支持标题更新检测)+ """保存新闻数据到 SQLite"""+ db_path = self._get_db_path(data.date)+ if not db_path.exists():+ # 确保目录存在+ db_path.parent.mkdir(parents=True, exist_ok=True)- Args:- data: 新闻数据-- Returns:- 是否保存成功- """- try:- conn = self._get_connection(data.date)- cursor = conn.cursor()-- # 获取配置时区的当前时间- now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")-- # 首先同步平台信息到 platforms 表- for source_id, source_name in data.id_to_name.items():- cursor.execute("""- INSERT INTO platforms (id, name, updated_at)- VALUES (?, ?, ?)- ON CONFLICT(id) DO UPDATE SET- name = excluded.name,- updated_at = excluded.updated_at- """, (source_id, source_name, now_str))-- # 统计计数器- new_count = 0- updated_count = 0- title_changed_count = 0- success_sources = []-- for source_id, news_list in data.items.items():- success_sources.append(source_id)-- for item in news_list:- try:- # 标准化 URL(去除动态参数,如微博的 band_rank)- normalized_url = normalize_url(item.url, source_id) if item.url else ""-- # 检查是否已存在(通过标准化 URL + platform_id)- if normalized_url:- cursor.execute("""- SELECT id, title FROM news_items- WHERE url = ? AND platform_id = ?- """, (normalized_url, source_id))- existing = cursor.fetchone()-- if existing:- # 已存在,更新记录- existing_id, existing_title = existing-- # 检查标题是否变化- if existing_title != item.title:- # 记录标题变更- cursor.execute("""- INSERT INTO title_changes- (news_item_id, old_title, new_title, changed_at)- VALUES (?, ?, ?, ?)- """, (existing_id, existing_title, item.title, now_str))- title_changed_count += 1-- # 记录排名历史- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (existing_id, item.rank, data.crawl_time, now_str))-- # 更新现有记录- cursor.execute("""- UPDATE news_items SET- title = ?,- rank = ?,- mobile_url = ?,- last_crawl_time = ?,- crawl_count = crawl_count + 1,- updated_at = ?- WHERE id = ?- """, (item.title, item.rank, item.mobile_url,- data.crawl_time, now_str, existing_id))- updated_count += 1- else:- # 不存在,插入新记录(存储标准化后的 URL)- cursor.execute("""- INSERT INTO news_items- (title, platform_id, rank, url, mobile_url,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, source_id, item.rank, normalized_url,- item.mobile_url, data.crawl_time, data.crawl_time,- now_str, now_str))- new_id = cursor.lastrowid- # 记录初始排名- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (new_id, item.rank, data.crawl_time, now_str))- new_count += 1- else:- # URL 为空的情况,直接插入(不做去重)- cursor.execute("""- INSERT INTO news_items- (title, platform_id, rank, url, mobile_url,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, source_id, item.rank, "",- item.mobile_url, data.crawl_time, data.crawl_time,- now_str, now_str))- new_id = cursor.lastrowid- # 记录初始排名- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (new_id, item.rank, data.crawl_time, now_str))- new_count += 1-- except sqlite3.Error as e:- print(f"保存新闻条目失败 [{item.title[:30]}...]: {e}")-- total_items = new_count + updated_count-- # 记录抓取信息- cursor.execute("""- INSERT OR REPLACE INTO crawl_records- (crawl_time, total_items, created_at)- VALUES (?, ?, ?)- """, (data.crawl_time, total_items, now_str))-- # 获取刚插入的 crawl_record 的 ID- cursor.execute("""- SELECT id FROM crawl_records WHERE crawl_time = ?- """, (data.crawl_time,))- record_row = cursor.fetchone()- if record_row:- crawl_record_id = record_row[0]-- # 记录成功的来源- for source_id in success_sources:- cursor.execute("""- INSERT OR REPLACE INTO crawl_source_status- (crawl_record_id, platform_id, status)- VALUES (?, ?, 'success')- """, (crawl_record_id, source_id))-- # 记录失败的来源- for failed_id in data.failed_ids:- # 确保失败的平台也在 platforms 表中- cursor.execute("""- INSERT OR IGNORE INTO platforms (id, name, updated_at)- VALUES (?, ?, ?)- """, (failed_id, failed_id, now_str))-- cursor.execute("""- INSERT OR REPLACE INTO crawl_source_status- (crawl_record_id, platform_id, status)- VALUES (?, ?, 'failed')- """, (crawl_record_id, failed_id))-- conn.commit()+ success, new_count, updated_count, title_changed_count, off_list_count = \+ self._save_news_data_impl(data, "[本地存储]")+ if success:# 输出详细的存储统计日志log_parts = [f"[本地存储] 处理完成:新增 {new_count} 条"]if updated_count > 0:log_parts.append(f"更新 {updated_count} 条")if title_changed_count > 0:log_parts.append(f"标题变更 {title_changed_count} 条")+ if off_list_count > 0:+ log_parts.append(f"脱榜 {off_list_count} 条")print(",".join(log_parts))- return True-- except Exception as e:- print(f"[本地存储] 保存失败: {e}")- return False+ return successdef get_today_all_data(self, date: Optional[str] = None) -> Optional[NewsData]:- """- 获取指定日期的所有新闻数据(合并后)+ """获取指定日期的所有新闻数据(合并后)"""+ db_path = self._get_db_path(date)+ if not db_path.exists():+ return None+ return self._get_today_all_data_impl(date)- Args:- date: 日期字符串,默认为今天+ def get_latest_crawl_data(self, date: Optional[str] = None) -> Optional[NewsData]:+ """获取最新一次抓取的数据"""+ db_path = self._get_db_path(date)+ if not db_path.exists():+ return None+ return self._get_latest_crawl_data_impl(date)- Returns:- 合并后的新闻数据- """- try:- db_path = self._get_db_path(date)- if not db_path.exists():- return None-- conn = self._get_connection(date)- cursor = conn.cursor()-- # 获取所有新闻数据(包含 id 用于查询排名历史)- cursor.execute("""- SELECT n.id, n.title, n.platform_id, p.name as platform_name,- n.rank, n.url, n.mobile_url,- n.first_crawl_time, n.last_crawl_time, n.crawl_count- FROM news_items n- LEFT JOIN platforms p ON n.platform_id = p.id- ORDER BY n.platform_id, n.last_crawl_time- """)-- rows = cursor.fetchall()- if not rows:- return None-- # 收集所有 news_item_id- news_ids = [row[0] for row in rows]-- # 批量查询排名历史- rank_history_map: Dict[int, List[int]] = {}- if news_ids:- placeholders = ",".join("?" * len(news_ids))- cursor.execute(f"""- SELECT news_item_id, rank FROM rank_history- WHERE news_item_id IN ({placeholders})- ORDER BY news_item_id, crawl_time- """, news_ids)- for rh_row in cursor.fetchall():- news_id, rank = rh_row[0], rh_row[1]- if news_id not in rank_history_map:- rank_history_map[news_id] = []- if rank not in rank_history_map[news_id]:- rank_history_map[news_id].append(rank)-- # 按 platform_id 分组- items: Dict[str, List[NewsItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- news_id = row[0]- platform_id = row[2]- title = row[1]- platform_name = row[3] or platform_id-- id_to_name[platform_id] = platform_name-- if platform_id not in items:- items[platform_id] = []-- # 获取排名历史,如果没有则使用当前排名- ranks = rank_history_map.get(news_id, [row[4]])-- items[platform_id].append(NewsItem(- title=title,- source_id=platform_id,- source_name=platform_name,- rank=row[4],- url=row[5] or "",- mobile_url=row[6] or "",- crawl_time=row[8], # last_crawl_time- ranks=ranks,- first_time=row[7], # first_crawl_time- last_time=row[8], # last_crawl_time- count=row[9], # crawl_count- ))-- final_items = items-- # 获取失败的来源- cursor.execute("""- SELECT DISTINCT css.platform_id- FROM crawl_source_status css- JOIN crawl_records cr ON css.crawl_record_id = cr.id- WHERE css.status = 'failed'- """)- failed_ids = [row[0] for row in cursor.fetchall()]-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- crawl_time = time_row[0] if time_row else self._format_time_filename()-- return NewsData(- date=crawl_date,- crawl_time=crawl_time,- items=final_items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )+ def detect_new_titles(self, current_data: NewsData) -> Dict[str, Dict]:+ """检测新增的标题"""+ return self._detect_new_titles_impl(current_data)- except Exception as e:- print(f"[本地存储] 读取数据失败: {e}")- return None+ def is_first_crawl_today(self, date: Optional[str] = None) -> bool:+ """检查是否是当天第一次抓取"""+ db_path = self._get_db_path(date)+ if not db_path.exists():+ return True+ return self._is_first_crawl_today_impl(date)- def get_latest_crawl_data(self, date: Optional[str] = None) -> Optional[NewsData]:- """- 获取最新一次抓取的数据+ def get_crawl_times(self, date: Optional[str] = None) -> List[str]:+ """获取指定日期的所有抓取时间列表"""+ db_path = self._get_db_path(date)+ if not db_path.exists():+ return []+ return self._get_crawl_times_impl(date)- Args:- date: 日期字符串,默认为今天+ def has_pushed_today(self, date: Optional[str] = None) -> bool:+ """检查指定日期是否已推送过"""+ return self._has_pushed_today_impl(date)- Returns:- 最新抓取的新闻数据- """- try:- db_path = self._get_db_path(date)- if not db_path.exists():- return None-- conn = self._get_connection(date)- cursor = conn.cursor()-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- if not time_row:- return None-- latest_time = time_row[0]-- # 获取该时间的新闻数据(包含 id 用于查询排名历史)- cursor.execute("""- SELECT n.id, n.title, n.platform_id, p.name as platform_name,- n.rank, n.url, n.mobile_url,- n.first_crawl_time, n.last_crawl_time, n.crawl_count- FROM news_items n- LEFT JOIN platforms p ON n.platform_id = p.id- WHERE n.last_crawl_time = ?- """, (latest_time,))-- rows = cursor.fetchall()- if not rows:- return None-- # 收集所有 news_item_id- news_ids = [row[0] for row in rows]-- # 批量查询排名历史- rank_history_map: Dict[int, List[int]] = {}- if news_ids:- placeholders = ",".join("?" * len(news_ids))- cursor.execute(f"""- SELECT news_item_id, rank FROM rank_history- WHERE news_item_id IN ({placeholders})- ORDER BY news_item_id, crawl_time- """, news_ids)- for rh_row in cursor.fetchall():- news_id, rank = rh_row[0], rh_row[1]- if news_id not in rank_history_map:- rank_history_map[news_id] = []- if rank not in rank_history_map[news_id]:- rank_history_map[news_id].append(rank)-- items: Dict[str, List[NewsItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- news_id = row[0]- platform_id = row[2]- platform_name = row[3] or platform_id- id_to_name[platform_id] = platform_name-- if platform_id not in items:- items[platform_id] = []-- # 获取排名历史,如果没有则使用当前排名- ranks = rank_history_map.get(news_id, [row[4]])-- items[platform_id].append(NewsItem(- title=row[1],- source_id=platform_id,- source_name=platform_name,- rank=row[4],- url=row[5] or "",- mobile_url=row[6] or "",- crawl_time=row[8], # last_crawl_time- ranks=ranks,- first_time=row[7], # first_crawl_time- last_time=row[8], # last_crawl_time- count=row[9], # crawl_count- ))-- # 获取失败的来源(针对最新一次抓取)- cursor.execute("""- SELECT css.platform_id- FROM crawl_source_status css- JOIN crawl_records cr ON css.crawl_record_id = cr.id- WHERE cr.crawl_time = ? AND css.status = 'failed'- """, (latest_time,))-- failed_ids = [row[0] for row in cursor.fetchall()]-- return NewsData(- date=crawl_date,- crawl_time=latest_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )+ def record_push(self, report_type: str, date: Optional[str] = None) -> bool:+ """记录推送"""+ success = self._record_push_impl(report_type, date)+ if success:+ now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")+ print(f"[本地存储] 推送记录已保存: {report_type} at {now_str}")+ return success- except Exception as e:- print(f"[本地存储] 获取最新数据失败: {e}")- return None+ # ========================================+ # RSS 数据存储方法+ # ========================================- def detect_new_titles(self, current_data: NewsData) -> Dict[str, Dict]:- """- 检测新增的标题+ def save_rss_data(self, data: RSSData) -> bool:+ """保存 RSS 数据到 SQLite"""+ success, new_count, updated_count = self._save_rss_data_impl(data, "[本地存储]")- 该方法比较当前抓取数据与历史数据,找出新增的标题。- 关键逻辑:只有在历史批次中从未出现过的标题才算新增。+ if success:+ # 输出统计日志+ log_parts = [f"[本地存储] RSS 处理完成:新增 {new_count} 条"]+ if updated_count > 0:+ log_parts.append(f"更新 {updated_count} 条")+ print(",".join(log_parts))- Args:- current_data: 当前抓取的数据+ return success- Returns:- 新增的标题数据 {source_id: {title: NewsItem}}- """- try:- # 获取历史数据- historical_data = self.get_today_all_data(current_data.date)-- if not historical_data:- # 没有历史数据,所有都是新的- new_titles = {}- for source_id, news_list in current_data.items.items():- new_titles[source_id] = {item.title: item for item in news_list}- return new_titles-- # 获取当前批次时间- current_time = current_data.crawl_time-- # 收集历史标题(first_time < current_time 的标题)- # 这样可以正确处理同一标题因 URL 变化而产生多条记录的情况- historical_titles: Dict[str, set] = {}- for source_id, news_list in historical_data.items.items():- historical_titles[source_id] = set()- for item in news_list:- first_time = getattr(item, 'first_time', item.crawl_time)- if first_time < current_time:- historical_titles[source_id].add(item.title)-- # 检查是否有历史数据- has_historical_data = any(len(titles) > 0 for titles in historical_titles.values())- if not has_historical_data:- # 第一次抓取,没有"新增"概念- return {}-- # 检测新增- new_titles = {}- for source_id, news_list in current_data.items.items():- hist_set = historical_titles.get(source_id, set())- for item in news_list:- if item.title not in hist_set:- if source_id not in new_titles:- new_titles[source_id] = {}- new_titles[source_id][item.title] = item-- return new_titles+ def get_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:+ """获取指定日期的所有 RSS 数据"""+ return self._get_rss_data_impl(date)- except Exception as e:- print(f"[本地存储] 检测新标题失败: {e}")- return {}+ def detect_new_rss_items(self, current_data: RSSData) -> Dict[str, List[RSSItem]]:+ """检测新增的 RSS 条目"""+ return self._detect_new_rss_items_impl(current_data)++ def get_latest_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:+ """获取最新一次抓取的 RSS 数据"""+ db_path = self._get_db_path(date, db_type="rss")+ if not db_path.exists():+ return None+ return self._get_latest_rss_data_impl(date)++ # ========================================+ # 本地特有功能:TXT/HTML 快照+ # ========================================def save_txt_snapshot(self, data: NewsData) -> Optional[str]:"""@@ -712,67 +318,9 @@ class LocalStorageBackend(StorageBackend):print(f"[本地存储] 保存 HTML 报告失败: {e}")return None- def is_first_crawl_today(self, date: Optional[str] = None) -> bool:- """- 检查是否是当天第一次抓取-- Args:- date: 日期字符串,默认为今天-- Returns:- 是否是第一次抓取- """- try:- db_path = self._get_db_path(date)- if not db_path.exists():- return True-- conn = self._get_connection(date)- cursor = conn.cursor()-- cursor.execute("""- SELECT COUNT(*) as count FROM crawl_records- """)-- row = cursor.fetchone()- count = row[0] if row else 0-- # 如果只有一条或没有记录,视为第一次抓取- return count <= 1-- except Exception as e:- print(f"[本地存储] 检查首次抓取失败: {e}")- return True-- def get_crawl_times(self, date: Optional[str] = None) -> List[str]:- """- 获取指定日期的所有抓取时间列表-- Args:- date: 日期字符串,默认为今天-- Returns:- 抓取时间列表(按时间排序)- """- try:- db_path = self._get_db_path(date)- if not db_path.exists():- return []-- conn = self._get_connection(date)- cursor = conn.cursor()-- cursor.execute("""- SELECT crawl_time FROM crawl_records- ORDER BY crawl_time- """)-- rows = cursor.fetchall()- return [row[0] for row in rows]-- except Exception as e:- print(f"[本地存储] 获取抓取时间列表失败: {e}")- return []+ # ========================================+ # 本地特有功能:资源清理+ # ========================================def cleanup(self) -> None:"""清理资源(关闭数据库连接)"""@@ -808,27 +356,17 @@ class LocalStorageBackend(StorageBackend):cutoff_date = self._get_configured_time() - timedelta(days=retention_days)def parse_date_from_name(name: str) -> Optional[datetime]:- """从文件名或目录名解析日期"""+ """从文件名或目录名解析日期 (ISO 格式: YYYY-MM-DD)"""# 移除 .db 后缀name = name.replace('.db', '')try:- # ISO 格式: YYYY-MM-DDdate_match = re.match(r'(\d{4})-(\d{2})-(\d{2})', name)if date_match:return datetime(int(date_match.group(1)),int(date_match.group(2)),int(date_match.group(3)),- tzinfo=pytz.timezone("Asia/Shanghai")- )- # 旧中文格式: YYYY年MM月DD日- date_match = re.match(r'(\d{4})年(\d{2})月(\d{2})日', name)- if date_match:- return datetime(- int(date_match.group(1)),- int(date_match.group(2)),- int(date_match.group(3)),- tzinfo=pytz.timezone("Asia/Shanghai")+ tzinfo=pytz.timezone(self.timezone))except Exception:pass@@ -892,449 +430,6 @@ class LocalStorageBackend(StorageBackend):print(f"[本地存储] 清理过期数据失败: {e}")return deleted_count- def has_pushed_today(self, date: Optional[str] = None) -> bool:- """- 检查指定日期是否已推送过-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 是否已推送- """- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- target_date = self._format_date_folder(date)-- cursor.execute("""- SELECT pushed FROM push_records WHERE date = ?- """, (target_date,))-- row = cursor.fetchone()- if row:- return bool(row[0])- return False-- except Exception as e:- print(f"[本地存储] 检查推送记录失败: {e}")- return False-- def record_push(self, report_type: str, date: Optional[str] = None) -> bool:- """- 记录推送-- Args:- report_type: 报告类型- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 是否记录成功- """- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- target_date = self._format_date_folder(date)- now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")-- cursor.execute("""- INSERT INTO push_records (date, pushed, push_time, report_type, created_at)- VALUES (?, 1, ?, ?, ?)- ON CONFLICT(date) DO UPDATE SET- pushed = 1,- push_time = excluded.push_time,- report_type = excluded.report_type- """, (target_date, now_str, report_type, now_str))-- conn.commit()-- print(f"[本地存储] 推送记录已保存: {report_type} at {now_str}")- return True-- except Exception as e:- print(f"[本地存储] 记录推送失败: {e}")- return False-- # ========================================- # RSS 数据存储方法- # ========================================-- def save_rss_data(self, data: RSSData) -> bool:- """- 保存 RSS 数据到 SQLite(以 URL 为唯一标识)-- Args:- data: RSS 数据-- Returns:- 是否保存成功- """- try:- conn = self._get_connection(data.date, db_type="rss")- cursor = conn.cursor()-- now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")-- # 同步 RSS 源信息到 rss_feeds 表- for feed_id, feed_name in data.id_to_name.items():- cursor.execute("""- INSERT INTO rss_feeds (id, name, updated_at)- VALUES (?, ?, ?)- ON CONFLICT(id) DO UPDATE SET- name = excluded.name,- updated_at = excluded.updated_at- """, (feed_id, feed_name, now_str))-- # 统计计数器- new_count = 0- updated_count = 0-- for feed_id, rss_list in data.items.items():- for item in rss_list:- try:- # 检查是否已存在(通过 URL + feed_id)- if item.url:- cursor.execute("""- SELECT id, title FROM rss_items- WHERE url = ? AND feed_id = ?- """, (item.url, feed_id))- existing = cursor.fetchone()-- if existing:- # 已存在,更新记录- existing_id = existing[0]- cursor.execute("""- UPDATE rss_items SET- title = ?,- published_at = ?,- summary = ?,- author = ?,- last_crawl_time = ?,- crawl_count = crawl_count + 1,- updated_at = ?- WHERE id = ?- """, (item.title, item.published_at, item.summary,- item.author, data.crawl_time, now_str, existing_id))- updated_count += 1- else:- # 不存在,插入新记录- cursor.execute("""- INSERT INTO rss_items- (title, feed_id, url, published_at, summary, author,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, feed_id, item.url, item.published_at,- item.summary, item.author, data.crawl_time,- data.crawl_time, now_str, now_str))- new_count += 1- else:- # URL 为空,直接插入- cursor.execute("""- INSERT INTO rss_items- (title, feed_id, url, published_at, summary, author,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, feed_id, "", item.published_at,- item.summary, item.author, data.crawl_time,- data.crawl_time, now_str, now_str))- new_count += 1-- except sqlite3.Error as e:- print(f"[本地存储] 保存 RSS 条目失败 [{item.title[:30]}...]: {e}")-- total_items = new_count + updated_count-- # 记录抓取信息- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_records- (crawl_time, total_items, created_at)- VALUES (?, ?, ?)- """, (data.crawl_time, total_items, now_str))-- # 记录抓取状态- cursor.execute("""- SELECT id FROM rss_crawl_records WHERE crawl_time = ?- """, (data.crawl_time,))- record_row = cursor.fetchone()- if record_row:- crawl_record_id = record_row[0]-- # 记录成功的源- for feed_id in data.items.keys():- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_status- (crawl_record_id, feed_id, status)- VALUES (?, ?, 'success')- """, (crawl_record_id, feed_id))-- # 记录失败的源- for failed_id in data.failed_ids:- cursor.execute("""- INSERT OR IGNORE INTO rss_feeds (id, name, updated_at)- VALUES (?, ?, ?)- """, (failed_id, failed_id, now_str))-- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_status- (crawl_record_id, feed_id, status)- VALUES (?, ?, 'failed')- """, (crawl_record_id, failed_id))-- conn.commit()-- # 输出统计日志- log_parts = [f"[本地存储] RSS 处理完成:新增 {new_count} 条"]- if updated_count > 0:- log_parts.append(f"更新 {updated_count} 条")- print(",".join(log_parts))-- return True-- except Exception as e:- print(f"[本地存储] 保存 RSS 数据失败: {e}")- return False-- def get_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:- """- 获取指定日期的所有 RSS 数据-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- RSSData 对象,如果没有数据返回 None- """- try:- conn = self._get_connection(date, db_type="rss")- cursor = conn.cursor()-- # 获取所有 RSS 数据- cursor.execute("""- SELECT i.id, i.title, i.feed_id, f.name as feed_name,- i.url, i.published_at, i.summary, i.author,- i.first_crawl_time, i.last_crawl_time, i.crawl_count- FROM rss_items i- LEFT JOIN rss_feeds f ON i.feed_id = f.id- ORDER BY i.published_at DESC- """)-- rows = cursor.fetchall()- if not rows:- return None-- items: Dict[str, List[RSSItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- feed_id = row[2]- feed_name = row[3] or feed_id-- id_to_name[feed_id] = feed_name-- if feed_id not in items:- items[feed_id] = []-- items[feed_id].append(RSSItem(- title=row[1],- feed_id=feed_id,- feed_name=feed_name,- url=row[4] or "",- published_at=row[5] or "",- summary=row[6] or "",- author=row[7] or "",- crawl_time=row[9],- first_time=row[8],- last_time=row[9],- count=row[10],- ))-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM rss_crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)- time_row = cursor.fetchone()- crawl_time = time_row[0] if time_row else self._format_time_filename()-- # 获取失败的源- cursor.execute("""- SELECT DISTINCT cs.feed_id- FROM rss_crawl_status cs- JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id- WHERE cs.status = 'failed'- """)- failed_ids = [row[0] for row in cursor.fetchall()]-- return RSSData(- date=crawl_date,- crawl_time=crawl_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )-- except Exception as e:- print(f"[本地存储] 读取 RSS 数据失败: {e}")- return None-- def detect_new_rss_items(self, current_data: RSSData) -> Dict[str, List[RSSItem]]:- """- 检测新增的 RSS 条目(增量模式)-- 该方法比较当前抓取数据与历史数据,找出新增的 RSS 条目。- 关键逻辑:只有在历史批次中从未出现过的 URL 才算新增。-- Args:- current_data: 当前抓取的 RSS 数据-- Returns:- 新增的 RSS 条目 {feed_id: [RSSItem, ...]}- """- try:- # 获取历史数据- historical_data = self.get_rss_data(current_data.date)-- if not historical_data:- # 没有历史数据,所有都是新的- return current_data.items.copy()-- # 获取当前批次时间- current_time = current_data.crawl_time-- # 收集历史 URL(first_time < current_time 的条目)- historical_urls: Dict[str, set] = {}- for feed_id, rss_list in historical_data.items.items():- historical_urls[feed_id] = set()- for item in rss_list:- first_time = getattr(item, 'first_time', item.crawl_time)- if first_time < current_time:- if item.url:- historical_urls[feed_id].add(item.url)-- # 检查是否有历史数据- has_historical_data = any(len(urls) > 0 for urls in historical_urls.values())- if not has_historical_data:- # 第一次抓取,没有"新增"概念- return {}-- # 检测新增- new_items: Dict[str, List[RSSItem]] = {}- for feed_id, rss_list in current_data.items.items():- hist_set = historical_urls.get(feed_id, set())- for item in rss_list:- # 通过 URL 判断是否新增- if item.url and item.url not in hist_set:- if feed_id not in new_items:- new_items[feed_id] = []- new_items[feed_id].append(item)-- return new_items-- except Exception as e:- print(f"[本地存储] 检测新 RSS 条目失败: {e}")- return {}-- def get_latest_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:- """- 获取最新一次抓取的 RSS 数据(当前榜单模式)-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 最新抓取的 RSS 数据,如果没有数据返回 None- """- try:- db_path = self._get_db_path(date, db_type="rss")- if not db_path.exists():- return None-- conn = self._get_connection(date, db_type="rss")- cursor = conn.cursor()-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM rss_crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- if not time_row:- return None-- latest_time = time_row[0]-- # 获取该时间的 RSS 数据- cursor.execute("""- SELECT i.id, i.title, i.feed_id, f.name as feed_name,- i.url, i.published_at, i.summary, i.author,- i.first_crawl_time, i.last_crawl_time, i.crawl_count- FROM rss_items i- LEFT JOIN rss_feeds f ON i.feed_id = f.id- WHERE i.last_crawl_time = ?- ORDER BY i.published_at DESC- """, (latest_time,))-- rows = cursor.fetchall()- if not rows:- return None-- items: Dict[str, List[RSSItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- feed_id = row[2]- feed_name = row[3] or feed_id-- id_to_name[feed_id] = feed_name-- if feed_id not in items:- items[feed_id] = []-- items[feed_id].append(RSSItem(- title=row[1],- feed_id=feed_id,- feed_name=feed_name,- url=row[4] or "",- published_at=row[5] or "",- summary=row[6] or "",- author=row[7] or "",- crawl_time=row[9],- first_time=row[8],- last_time=row[9],- count=row[10],- ))-- # 获取失败的源(针对最新一次抓取)- cursor.execute("""- SELECT cs.feed_id- FROM rss_crawl_status cs- JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id- WHERE cr.crawl_time = ? AND cs.status = 'failed'- """, (latest_time,))-- failed_ids = [row[0] for row in cursor.fetchall()]-- return RSSData(- date=crawl_date,- crawl_time=latest_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )-- except Exception as e:- print(f"[本地存储] 获取最新 RSS 数据失败: {e}")- return None-def __del__(self):"""析构函数,确保关闭连接"""self.cleanup()diff --git a/trendradar/storage/remote.py b/trendradar/storage/remote.pyindex 45a2431ec0..7181133425 100644--- a/trendradar/storage/remote.py+++ b/trendradar/storage/remote.py@@ -29,15 +29,15 @@ except ImportError:ClientError = Exceptionfrom trendradar.storage.base import StorageBackend, NewsItem, NewsData, RSSItem, RSSData+from trendradar.storage.sqlite_mixin import SQLiteStorageMixinfrom trendradar.utils.time import (get_configured_time,format_date_folder,format_time_filename,)-from trendradar.utils.url import normalize_url-class RemoteStorageBackend(StorageBackend):+class RemoteStorageBackend(SQLiteStorageMixin, StorageBackend):"""远程云存储后端(S3 兼容协议)@@ -128,6 +128,10 @@ class RemoteStorageBackend(StorageBackend):def supports_txt(self) -> bool:return self.enable_txt+ # ========================================+ # SQLiteStorageMixin 抽象方法实现+ # ========================================+def _get_configured_time(self) -> datetime:"""获取配置时区的当前时间"""return get_configured_time(self.timezone)@@ -323,475 +327,138 @@ class RemoteStorageBackend(StorageBackend):return self._db_connections[db_path]- def _get_schema_path(self, db_type: str = "news") -> Path:- """- 获取 schema.sql 文件路径-- Args:- db_type: 数据库类型 ("news" 或 "rss")+ # ========================================+ # StorageBackend 接口实现(委托给 mixin + 上传)+ # ========================================- Returns:- schema 文件路径+ def save_news_data(self, data: NewsData) -> bool:"""- if db_type == "rss":- return Path(__file__).parent / "rss_schema.sql"- return Path(__file__).parent / "schema.sql"+ 保存新闻数据到远程存储- def _init_tables(self, conn: sqlite3.Connection, db_type: str = "news") -> None:+ 流程:下载现有数据库 → 插入/更新数据 → 上传回远程存储"""- 从 schema.sql 初始化数据库表结构+ # 查询已有记录数+ conn = self._get_connection(data.date)+ cursor = conn.cursor()+ cursor.execute("SELECT COUNT(*) as count FROM news_items")+ row = cursor.fetchone()+ existing_count = row[0] if row else 0+ if existing_count > 0:+ print(f"[远程存储] 已有 {existing_count} 条历史记录,将合并新数据")- Args:- conn: 数据库连接- db_type: 数据库类型 ("news" 或 "rss")- """- schema_path = self._get_schema_path(db_type)+ # 使用 mixin 的实现保存数据+ success, new_count, updated_count, title_changed_count, off_list_count = \+ self._save_news_data_impl(data, "[远程存储]")++ if not success:+ return False- if schema_path.exists():- with open(schema_path, "r", encoding="utf-8") as f:- schema_sql = f.read()- conn.executescript(schema_sql)+ # 查询合并后的总记录数+ cursor.execute("SELECT COUNT(*) as count FROM news_items")+ row = cursor.fetchone()+ final_count = row[0] if row else 0++ # 输出详细的存储统计日志+ log_parts = [f"[远程存储] 处理完成:新增 {new_count} 条"]+ if updated_count > 0:+ log_parts.append(f"更新 {updated_count} 条")+ if title_changed_count > 0:+ log_parts.append(f"标题变更 {title_changed_count} 条")+ if off_list_count > 0:+ log_parts.append(f"脱榜 {off_list_count} 条")+ log_parts.append(f"(去重后总计: {final_count} 条)")+ print(",".join(log_parts))++ # 上传到远程存储+ if self._upload_sqlite(data.date):+ print(f"[远程存储] 数据已同步到远程存储")+ return Trueelse:- raise FileNotFoundError(f"Schema file not found: {schema_path}")+ print(f"[远程存储] 上传远程存储失败")+ return False- conn.commit()+ def get_today_all_data(self, date: Optional[str] = None) -> Optional[NewsData]:+ """获取指定日期的所有新闻数据(合并后)"""+ return self._get_today_all_data_impl(date)- def save_news_data(self, data: NewsData) -> bool:- """- 保存新闻数据到远程存储(以 URL 为唯一标识,支持标题更新检测)+ def get_latest_crawl_data(self, date: Optional[str] = None) -> Optional[NewsData]:+ """获取最新一次抓取的数据"""+ return self._get_latest_crawl_data_impl(date)- 流程:下载现有数据库 → 插入/更新数据 → 上传回远程存储+ def detect_new_titles(self, current_data: NewsData) -> Dict[str, Dict]:+ """检测新增的标题"""+ return self._detect_new_titles_impl(current_data)- Args:- data: 新闻数据+ def is_first_crawl_today(self, date: Optional[str] = None) -> bool:+ """检查是否是当天第一次抓取"""+ return self._is_first_crawl_today_impl(date)- Returns:- 是否保存成功- """- try:- conn = self._get_connection(data.date)- cursor = conn.cursor()+ def has_pushed_today(self, date: Optional[str] = None) -> bool:+ """检查指定日期是否已推送过"""+ return self._has_pushed_today_impl(date)- # 查询已有记录数- cursor.execute("SELECT COUNT(*) as count FROM news_items")- row = cursor.fetchone()- existing_count = row[0] if row else 0- if existing_count > 0:- print(f"[远程存储] 已有 {existing_count} 条历史记录,将合并新数据")+ def record_push(self, report_type: str, date: Optional[str] = None) -> bool:+ """记录推送"""+ success = self._record_push_impl(report_type, date)- # 获取配置时区的当前时间+ if success:now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")+ print(f"[远程存储] 推送记录已保存: {report_type} at {now_str}")- # 首先同步平台信息到 platforms 表- for source_id, source_name in data.id_to_name.items():- cursor.execute("""- INSERT INTO platforms (id, name, updated_at)- VALUES (?, ?, ?)- ON CONFLICT(id) DO UPDATE SET- name = excluded.name,- updated_at = excluded.updated_at- """, (source_id, source_name, now_str))-- # 统计计数器- new_count = 0- updated_count = 0- title_changed_count = 0- success_sources = []-- for source_id, news_list in data.items.items():- success_sources.append(source_id)-- for item in news_list:- try:- # 标准化 URL(去除动态参数,如微博的 band_rank)- normalized_url = normalize_url(item.url, source_id) if item.url else ""-- # 检查是否已存在(通过标准化 URL + platform_id)- if normalized_url:- cursor.execute("""- SELECT id, title FROM news_items- WHERE url = ? AND platform_id = ?- """, (normalized_url, source_id))- existing = cursor.fetchone()-- if existing:- # 已存在,更新记录- existing_id, existing_title = existing-- # 检查标题是否变化- if existing_title != item.title:- # 记录标题变更- cursor.execute("""- INSERT INTO title_changes- (news_item_id, old_title, new_title, changed_at)- VALUES (?, ?, ?, ?)- """, (existing_id, existing_title, item.title, now_str))- title_changed_count += 1-- # 记录排名历史- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (existing_id, item.rank, data.crawl_time, now_str))-- # 更新现有记录- cursor.execute("""- UPDATE news_items SET- title = ?,- rank = ?,- mobile_url = ?,- last_crawl_time = ?,- crawl_count = crawl_count + 1,- updated_at = ?- WHERE id = ?- """, (item.title, item.rank, item.mobile_url,- data.crawl_time, now_str, existing_id))- updated_count += 1- else:- # 不存在,插入新记录(存储标准化后的 URL)- cursor.execute("""- INSERT INTO news_items- (title, platform_id, rank, url, mobile_url,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, source_id, item.rank, normalized_url,- item.mobile_url, data.crawl_time, data.crawl_time,- now_str, now_str))- new_id = cursor.lastrowid- # 记录初始排名- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (new_id, item.rank, data.crawl_time, now_str))- new_count += 1- else:- # URL 为空的情况,直接插入(不做去重)- cursor.execute("""- INSERT INTO news_items- (title, platform_id, rank, url, mobile_url,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, source_id, item.rank, "",- item.mobile_url, data.crawl_time, data.crawl_time,- now_str, now_str))- new_id = cursor.lastrowid- # 记录初始排名- cursor.execute("""- INSERT INTO rank_history- (news_item_id, rank, crawl_time, created_at)- VALUES (?, ?, ?, ?)- """, (new_id, item.rank, data.crawl_time, now_str))- new_count += 1-- except sqlite3.Error as e:- print(f"[远程存储] 保存新闻条目失败 [{item.title[:30]}...]: {e}")-- total_items = new_count + updated_count-- # 记录抓取信息- cursor.execute("""- INSERT OR REPLACE INTO crawl_records- (crawl_time, total_items, created_at)- VALUES (?, ?, ?)- """, (data.crawl_time, total_items, now_str))-- # 获取刚插入的 crawl_record 的 ID- cursor.execute("""- SELECT id FROM crawl_records WHERE crawl_time = ?- """, (data.crawl_time,))- record_row = cursor.fetchone()- if record_row:- crawl_record_id = record_row[0]-- # 记录成功的来源- for source_id in success_sources:- cursor.execute("""- INSERT OR REPLACE INTO crawl_source_status- (crawl_record_id, platform_id, status)- VALUES (?, ?, 'success')- """, (crawl_record_id, source_id))-- # 记录失败的来源- for failed_id in data.failed_ids:- # 确保失败的平台也在 platforms 表中- cursor.execute("""- INSERT OR IGNORE INTO platforms (id, name, updated_at)- VALUES (?, ?, ?)- """, (failed_id, failed_id, now_str))-- cursor.execute("""- INSERT OR REPLACE INTO crawl_source_status- (crawl_record_id, platform_id, status)- VALUES (?, ?, 'failed')- """, (crawl_record_id, failed_id))-- conn.commit()-- # 查询合并后的总记录数- cursor.execute("SELECT COUNT(*) as count FROM news_items")- row = cursor.fetchone()- final_count = row[0] if row else 0-- # 输出详细的存储统计日志- log_parts = [f"[远程存储] 处理完成:新增 {new_count} 条"]- if updated_count > 0:- log_parts.append(f"更新 {updated_count} 条")- if title_changed_count > 0:- log_parts.append(f"标题变更 {title_changed_count} 条")- log_parts.append(f"(去重后总计: {final_count} 条)")- print(",".join(log_parts))-- # 上传到远程存储- if self._upload_sqlite(data.date):- print(f"[远程存储] 数据已同步到远程存储")+ # 上传到远程存储 确保记录持久化+ if self._upload_sqlite(date):+ print(f"[远程存储] 推送记录已同步到远程存储")return Trueelse:- print(f"[远程存储] 上传远程存储失败")+ print(f"[远程存储] 推送记录同步到远程存储失败")return False- except Exception as e:- print(f"[远程存储] 保存失败: {e}")- return False+ return False- def get_today_all_data(self, date: Optional[str] = None) -> Optional[NewsData]:- """获取指定日期的所有新闻数据(合并后)"""- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- # 获取所有新闻数据(包含 id 用于查询排名历史)- cursor.execute("""- SELECT n.id, n.title, n.platform_id, p.name as platform_name,- n.rank, n.url, n.mobile_url,- n.first_crawl_time, n.last_crawl_time, n.crawl_count- FROM news_items n- LEFT JOIN platforms p ON n.platform_id = p.id- ORDER BY n.platform_id, n.last_crawl_time- """)-- rows = cursor.fetchall()- if not rows:- return None-- # 收集所有 news_item_id- news_ids = [row[0] for row in rows]-- # 批量查询排名历史- rank_history_map: Dict[int, List[int]] = {}- if news_ids:- placeholders = ",".join("?" * len(news_ids))- cursor.execute(f"""- SELECT news_item_id, rank FROM rank_history- WHERE news_item_id IN ({placeholders})- ORDER BY news_item_id, crawl_time- """, news_ids)- for rh_row in cursor.fetchall():- news_id, rank = rh_row[0], rh_row[1]- if news_id not in rank_history_map:- rank_history_map[news_id] = []- if rank not in rank_history_map[news_id]:- rank_history_map[news_id].append(rank)-- # 按 platform_id 分组- items: Dict[str, List[NewsItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- news_id = row[0]- platform_id = row[2]- title = row[1]- platform_name = row[3] or platform_id-- id_to_name[platform_id] = platform_name-- if platform_id not in items:- items[platform_id] = []-- # 获取排名历史,如果没有则使用当前排名- ranks = rank_history_map.get(news_id, [row[4]])-- items[platform_id].append(NewsItem(- title=title,- source_id=platform_id,- source_name=platform_name,- rank=row[4],- url=row[5] or "",- mobile_url=row[6] or "",- crawl_time=row[8], # last_crawl_time- ranks=ranks,- first_time=row[7], # first_crawl_time- last_time=row[8], # last_crawl_time- count=row[9], # crawl_count- ))-- final_items = items-- # 获取失败的来源- cursor.execute("""- SELECT DISTINCT css.platform_id- FROM crawl_source_status css- JOIN crawl_records cr ON css.crawl_record_id = cr.id- WHERE css.status = 'failed'- """)- failed_ids = [row[0] for row in cursor.fetchall()]-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- crawl_time = time_row[0] if time_row else self._format_time_filename()-- return NewsData(- date=crawl_date,- crawl_time=crawl_time,- items=final_items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )-- except Exception as e:- print(f"[远程存储] 读取数据失败: {e}")- return None+ # ========================================+ # RSS 数据存储方法+ # ========================================- def get_latest_crawl_data(self, date: Optional[str] = None) -> Optional[NewsData]:- """获取最新一次抓取的数据"""- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- if not time_row:- return None+ def save_rss_data(self, data: RSSData) -> bool:+ """+ 保存 RSS 数据到远程存储- latest_time = time_row[0]+ 流程:下载现有数据库 → 插入/更新数据 → 上传回远程存储+ """+ success, new_count, updated_count = self._save_rss_data_impl(data, "[远程存储]")- # 获取该时间的新闻数据,通过 JOIN 获取平台名称- cursor.execute("""- SELECT n.title, n.platform_id, p.name as platform_name,- n.rank, n.url, n.mobile_url,- n.first_crawl_time, n.last_crawl_time, n.crawl_count- FROM news_items n- LEFT JOIN platforms p ON n.platform_id = p.id- WHERE n.last_crawl_time = ?- """, (latest_time,))+ if not success:+ return False- rows = cursor.fetchall()- if not rows:- return None+ # 输出统计日志+ log_parts = [f"[远程存储] RSS 处理完成:新增 {new_count} 条"]+ if updated_count > 0:+ log_parts.append(f"更新 {updated_count} 条")+ print(",".join(log_parts))- items: Dict[str, List[NewsItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- platform_id = row[1]- platform_name = row[2] or platform_id- id_to_name[platform_id] = platform_name-- if platform_id not in items:- items[platform_id] = []-- items[platform_id].append(NewsItem(- title=row[0],- source_id=platform_id,- source_name=platform_name,- rank=row[3],- url=row[4] or "",- mobile_url=row[5] or "",- crawl_time=row[7], # last_crawl_time- ranks=[row[3]],- first_time=row[6], # first_crawl_time- last_time=row[7], # last_crawl_time- count=row[8], # crawl_count- ))-- # 获取失败的来源(针对最新一次抓取)- cursor.execute("""- SELECT css.platform_id- FROM crawl_source_status css- JOIN crawl_records cr ON css.crawl_record_id = cr.id- WHERE cr.crawl_time = ? AND css.status = 'failed'- """, (latest_time,))-- failed_ids = [row[0] for row in cursor.fetchall()]-- return NewsData(- date=crawl_date,- crawl_time=latest_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )+ # 上传到远程存储+ if self._upload_sqlite(data.date, db_type="rss"):+ print(f"[远程存储] RSS 数据已同步到远程存储")+ return True+ else:+ print(f"[远程存储] RSS 上传远程存储失败")+ return False- except Exception as e:- print(f"[远程存储] 获取最新数据失败: {e}")- return None+ def get_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:+ """获取指定日期的所有 RSS 数据"""+ return self._get_rss_data_impl(date)- def detect_new_titles(self, current_data: NewsData) -> Dict[str, Dict]:- """- 检测新增的标题+ def detect_new_rss_items(self, current_data: RSSData) -> Dict[str, List[RSSItem]]:+ """检测新增的 RSS 条目"""+ return self._detect_new_rss_items_impl(current_data)- 该方法比较当前抓取数据与历史数据,找出新增的标题。- 关键逻辑:只有在历史批次中从未出现过的标题才算新增。- """- try:- historical_data = self.get_today_all_data(current_data.date)-- if not historical_data:- new_titles = {}- for source_id, news_list in current_data.items.items():- new_titles[source_id] = {item.title: item for item in news_list}- return new_titles-- # 获取当前批次时间- current_time = current_data.crawl_time-- # 收集历史标题(first_time < current_time 的标题)- # 这样可以正确处理同一标题因 URL 变化而产生多条记录的情况- historical_titles: Dict[str, set] = {}- for source_id, news_list in historical_data.items.items():- historical_titles[source_id] = set()- for item in news_list:- first_time = getattr(item, 'first_time', item.crawl_time)- if first_time < current_time:- historical_titles[source_id].add(item.title)-- # 检查是否有历史数据- has_historical_data = any(len(titles) > 0 for titles in historical_titles.values())- if not has_historical_data:- # 第一次抓取,没有"新增"概念- return {}-- new_titles = {}- for source_id, news_list in current_data.items.items():- hist_set = historical_titles.get(source_id, set())- for item in news_list:- if item.title not in hist_set:- if source_id not in new_titles:- new_titles[source_id] = {}- new_titles[source_id][item.title] = item-- return new_titles+ def get_latest_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:+ """获取最新一次抓取的 RSS 数据"""+ return self._get_latest_rss_data_impl(date)- except Exception as e:- print(f"[远程存储] 检测新标题失败: {e}")- return {}+ # ========================================+ # 远程特有功能:TXT/HTML 快照(临时目录)+ # ========================================def save_txt_snapshot(self, data: NewsData) -> Optional[str]:"""保存 TXT 快照(远程存储模式下默认不支持)"""@@ -861,24 +528,9 @@ class RemoteStorageBackend(StorageBackend):print(f"[远程存储] 保存 HTML 报告失败: {e}")return None- def is_first_crawl_today(self, date: Optional[str] = None) -> bool:- """检查是否是当天第一次抓取"""- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- cursor.execute("""- SELECT COUNT(*) as count FROM crawl_records- """)-- row = cursor.fetchone()- count = row[0] if row else 0-- return count <= 1-- except Exception as e:- print(f"[远程存储] 检查首次抓取失败: {e}")- return True+ # ========================================+ # 远程特有功能:资源清理+ # ========================================def cleanup(self) -> None:"""清理资源(关闭连接和删除临时文件)"""@@ -946,30 +598,19 @@ class RemoteStorageBackend(StorageBackend):for obj in page['Contents']:key = obj['Key']- # 解析日期(格式: news/YYYY-MM-DD.db 或 news/YYYY年MM月DD日.db)+ # 解析日期(格式: news/YYYY-MM-DD.db)folder_date = None+ date_str = Nonetry:- # ISO 格式: news/YYYY-MM-DD.dbdate_match = re.match(r'news/(\d{4})-(\d{2})-(\d{2})\.db$', key)if date_match:folder_date = datetime(int(date_match.group(1)),int(date_match.group(2)),int(date_match.group(3)),- tzinfo=pytz.timezone("Asia/Shanghai")+ tzinfo=pytz.timezone(self.timezone))date_str = f"{date_match.group(1)}-{date_match.group(2)}-{date_match.group(3)}"- else:- # 旧中文格式: news/YYYY年MM月DD日.db- date_match = re.match(r'news/(\d{4})年(\d{2})月(\d{2})日\.db$', key)- if date_match:- folder_date = datetime(- int(date_match.group(1)),- int(date_match.group(2)),- int(date_match.group(3)),- tzinfo=pytz.timezone("Asia/Shanghai")- )- date_str = f"{date_match.group(1)}年{date_match.group(2)}月{date_match.group(3)}日"except Exception:continue@@ -1003,460 +644,6 @@ class RemoteStorageBackend(StorageBackend):print(f"[远程存储] 清理过期数据失败: {e}")return deleted_count- def has_pushed_today(self, date: Optional[str] = None) -> bool:- """- 检查指定日期是否已推送过-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 是否已推送- """- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- target_date = self._format_date_folder(date)-- cursor.execute("""- SELECT pushed FROM push_records WHERE date = ?- """, (target_date,))-- row = cursor.fetchone()- if row:- return bool(row[0])- return False-- except Exception as e:- print(f"[远程存储] 检查推送记录失败: {e}")- return False-- def record_push(self, report_type: str, date: Optional[str] = None) -> bool:- """- 记录推送-- Args:- report_type: 报告类型- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 是否记录成功- """- try:- conn = self._get_connection(date)- cursor = conn.cursor()-- target_date = self._format_date_folder(date)- now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")-- cursor.execute("""- INSERT INTO push_records (date, pushed, push_time, report_type, created_at)- VALUES (?, 1, ?, ?, ?)- ON CONFLICT(date) DO UPDATE SET- pushed = 1,- push_time = excluded.push_time,- report_type = excluded.report_type- """, (target_date, now_str, report_type, now_str))-- conn.commit()-- print(f"[远程存储] 推送记录已保存: {report_type} at {now_str}")-- # 上传到远程存储 确保记录持久化- if self._upload_sqlite(date):- print(f"[远程存储] 推送记录已同步到远程存储")- return True- else:- print(f"[远程存储] 推送记录同步到远程存储失败")- return False-- except Exception as e:- print(f"[远程存储] 记录推送失败: {e}")- return False-- # ========================================- # RSS 数据存储方法- # ========================================-- def save_rss_data(self, data: RSSData) -> bool:- """- 保存 RSS 数据到远程存储(以 URL 为唯一标识)-- 流程:下载现有数据库 → 插入/更新数据 → 上传回远程存储-- Args:- data: RSS 数据-- Returns:- 是否保存成功- """- try:- conn = self._get_connection(data.date, db_type="rss")- cursor = conn.cursor()-- now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")-- # 同步 RSS 源信息到 rss_feeds 表- for feed_id, feed_name in data.id_to_name.items():- cursor.execute("""- INSERT INTO rss_feeds (id, name, updated_at)- VALUES (?, ?, ?)- ON CONFLICT(id) DO UPDATE SET- name = excluded.name,- updated_at = excluded.updated_at- """, (feed_id, feed_name, now_str))-- # 统计计数器- new_count = 0- updated_count = 0-- for feed_id, rss_list in data.items.items():- for item in rss_list:- try:- # 检查是否已存在(通过 URL + feed_id)- if item.url:- cursor.execute("""- SELECT id, title FROM rss_items- WHERE url = ? AND feed_id = ?- """, (item.url, feed_id))- existing = cursor.fetchone()-- if existing:- # 已存在,更新记录- existing_id = existing[0]- cursor.execute("""- UPDATE rss_items SET- title = ?,- published_at = ?,- summary = ?,- author = ?,- last_crawl_time = ?,- crawl_count = crawl_count + 1,- updated_at = ?- WHERE id = ?- """, (item.title, item.published_at, item.summary,- item.author, data.crawl_time, now_str, existing_id))- updated_count += 1- else:- # 不存在,插入新记录- cursor.execute("""- INSERT INTO rss_items- (title, feed_id, url, published_at, summary, author,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, feed_id, item.url, item.published_at,- item.summary, item.author, data.crawl_time,- data.crawl_time, now_str, now_str))- new_count += 1- else:- # URL 为空,直接插入- cursor.execute("""- INSERT INTO rss_items- (title, feed_id, url, published_at, summary, author,- first_crawl_time, last_crawl_time, crawl_count,- created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)- """, (item.title, feed_id, "", item.published_at,- item.summary, item.author, data.crawl_time,- data.crawl_time, now_str, now_str))- new_count += 1-- except sqlite3.Error as e:- print(f"[远程存储] 保存 RSS 条目失败 [{item.title[:30]}...]: {e}")-- total_items = new_count + updated_count-- # 记录抓取信息- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_records- (crawl_time, total_items, created_at)- VALUES (?, ?, ?)- """, (data.crawl_time, total_items, now_str))-- # 记录抓取状态- cursor.execute("""- SELECT id FROM rss_crawl_records WHERE crawl_time = ?- """, (data.crawl_time,))- record_row = cursor.fetchone()- if record_row:- crawl_record_id = record_row[0]-- # 记录成功的源- for feed_id in data.items.keys():- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_status- (crawl_record_id, feed_id, status)- VALUES (?, ?, 'success')- """, (crawl_record_id, feed_id))-- # 记录失败的源- for failed_id in data.failed_ids:- cursor.execute("""- INSERT OR IGNORE INTO rss_feeds (id, name, updated_at)- VALUES (?, ?, ?)- """, (failed_id, failed_id, now_str))-- cursor.execute("""- INSERT OR REPLACE INTO rss_crawl_status- (crawl_record_id, feed_id, status)- VALUES (?, ?, 'failed')- """, (crawl_record_id, failed_id))-- conn.commit()-- # 输出统计日志- log_parts = [f"[远程存储] RSS 处理完成:新增 {new_count} 条"]- if updated_count > 0:- log_parts.append(f"更新 {updated_count} 条")- print(",".join(log_parts))-- # 上传到远程存储- if self._upload_sqlite(data.date, db_type="rss"):- print(f"[远程存储] RSS 数据已同步到远程存储")- return True- else:- print(f"[远程存储] RSS 上传远程存储失败")- return False-- except Exception as e:- print(f"[远程存储] 保存 RSS 数据失败: {e}")- return False-- def get_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:- """- 获取指定日期的所有 RSS 数据-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- RSSData 对象,如果没有数据返回 None- """- try:- conn = self._get_connection(date, db_type="rss")- cursor = conn.cursor()-- # 获取所有 RSS 数据- cursor.execute("""- SELECT i.id, i.title, i.feed_id, f.name as feed_name,- i.url, i.published_at, i.summary, i.author,- i.first_crawl_time, i.last_crawl_time, i.crawl_count- FROM rss_items i- LEFT JOIN rss_feeds f ON i.feed_id = f.id- ORDER BY i.published_at DESC- """)-- rows = cursor.fetchall()- if not rows:- return None-- items: Dict[str, List[RSSItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- feed_id = row[2]- feed_name = row[3] or feed_id-- id_to_name[feed_id] = feed_name-- if feed_id not in items:- items[feed_id] = []-- items[feed_id].append(RSSItem(- title=row[1],- feed_id=feed_id,- feed_name=feed_name,- url=row[4] or "",- published_at=row[5] or "",- summary=row[6] or "",- author=row[7] or "",- crawl_time=row[9],- first_time=row[8],- last_time=row[9],- count=row[10],- ))-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM rss_crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)- time_row = cursor.fetchone()- crawl_time = time_row[0] if time_row else self._format_time_filename()-- # 获取失败的源- cursor.execute("""- SELECT DISTINCT cs.feed_id- FROM rss_crawl_status cs- JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id- WHERE cs.status = 'failed'- """)- failed_ids = [row[0] for row in cursor.fetchall()]-- return RSSData(- date=crawl_date,- crawl_time=crawl_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )-- except Exception as e:- print(f"[远程存储] 读取 RSS 数据失败: {e}")- return None-- def detect_new_rss_items(self, current_data: RSSData) -> Dict[str, List[RSSItem]]:- """- 检测新增的 RSS 条目(增量模式)-- 该方法比较当前抓取数据与历史数据,找出新增的 RSS 条目。- 关键逻辑:只有在历史批次中从未出现过的 URL 才算新增。-- Args:- current_data: 当前抓取的 RSS 数据-- Returns:- 新增的 RSS 条目 {feed_id: [RSSItem, ...]}- """- try:- # 获取历史数据- historical_data = self.get_rss_data(current_data.date)-- if not historical_data:- # 没有历史数据,所有都是新的- return current_data.items.copy()-- # 获取当前批次时间- current_time = current_data.crawl_time-- # 收集历史 URL(first_time < current_time 的条目)- historical_urls: Dict[str, set] = {}- for feed_id, rss_list in historical_data.items.items():- historical_urls[feed_id] = set()- for item in rss_list:- first_time = getattr(item, 'first_time', item.crawl_time)- if first_time < current_time:- if item.url:- historical_urls[feed_id].add(item.url)-- # 检查是否有历史数据- has_historical_data = any(len(urls) > 0 for urls in historical_urls.values())- if not has_historical_data:- # 第一次抓取,没有"新增"概念- return {}-- # 检测新增- new_items: Dict[str, List[RSSItem]] = {}- for feed_id, rss_list in current_data.items.items():- hist_set = historical_urls.get(feed_id, set())- for item in rss_list:- # 通过 URL 判断是否新增- if item.url and item.url not in hist_set:- if feed_id not in new_items:- new_items[feed_id] = []- new_items[feed_id].append(item)-- return new_items-- except Exception as e:- print(f"[远程存储] 检测新 RSS 条目失败: {e}")- return {}-- def get_latest_rss_data(self, date: Optional[str] = None) -> Optional[RSSData]:- """- 获取最新一次抓取的 RSS 数据(当前榜单模式)-- Args:- date: 日期字符串(YYYY-MM-DD),默认为今天-- Returns:- 最新抓取的 RSS 数据,如果没有数据返回 None- """- try:- conn = self._get_connection(date, db_type="rss")- cursor = conn.cursor()-- # 获取最新的抓取时间- cursor.execute("""- SELECT crawl_time FROM rss_crawl_records- ORDER BY crawl_time DESC- LIMIT 1- """)-- time_row = cursor.fetchone()- if not time_row:- return None-- latest_time = time_row[0]-- # 获取该时间的 RSS 数据- cursor.execute("""- SELECT i.id, i.title, i.feed_id, f.name as feed_name,- i.url, i.published_at, i.summary, i.author,- i.first_crawl_time, i.last_crawl_time, i.crawl_count- FROM rss_items i- LEFT JOIN rss_feeds f ON i.feed_id = f.id- WHERE i.last_crawl_time = ?- ORDER BY i.published_at DESC- """, (latest_time,))-- rows = cursor.fetchall()- if not rows:- return None-- items: Dict[str, List[RSSItem]] = {}- id_to_name: Dict[str, str] = {}- crawl_date = self._format_date_folder(date)-- for row in rows:- feed_id = row[2]- feed_name = row[3] or feed_id-- id_to_name[feed_id] = feed_name-- if feed_id not in items:- items[feed_id] = []-- items[feed_id].append(RSSItem(- title=row[1],- feed_id=feed_id,- feed_name=feed_name,- url=row[4] or "",- published_at=row[5] or "",- summary=row[6] or "",- author=row[7] or "",- crawl_time=row[9],- first_time=row[8],- last_time=row[9],- count=row[10],- ))-- # 获取失败的源(针对最新一次抓取)- cursor.execute("""- SELECT cs.feed_id- FROM rss_crawl_status cs- JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id- WHERE cr.crawl_time = ? AND cs.status = 'failed'- """, (latest_time,))-- failed_ids = [row[0] for row in cursor.fetchall()]-- return RSSData(- date=crawl_date,- crawl_time=latest_time,- items=items,- id_to_name=id_to_name,- failed_ids=failed_ids,- )-- except Exception as e:- print(f"[远程存储] 获取最新 RSS 数据失败: {e}")- return None-def __del__(self):"""析构函数"""# 检查 Python 是否正在关闭@@ -1468,6 +655,10 @@ class RemoteStorageBackend(StorageBackend):# Python 关闭时可能会出错,忽略即可pass+ # ========================================+ # 远程特有功能:数据拉取和列表+ # ========================================+def pull_recent_days(self, days: int, local_data_dir: str = "output") -> int:"""从远程拉取最近 N 天的数据到本地diff --git a/trendradar/storage/sqlite_mixin.py b/trendradar/storage/sqlite_mixin.pynew file mode 100644index 0000000000..754d33eb6a--- /dev/null+++ b/trendradar/storage/sqlite_mixin.py@@ -0,0 +1,1137 @@+# coding=utf-8+"""+SQLite 存储 Mixin++提供共用的 SQLite 数据库操作逻辑,供 LocalStorageBackend 和 RemoteStorageBackend 复用。+"""++import sqlite3+from abc import abstractmethod+from datetime import datetime+from pathlib import Path+from typing import Any, Dict, List, Optional++from trendradar.storage.base import NewsItem, NewsData, RSSItem, RSSData+from trendradar.utils.url import normalize_url+++class SQLiteStorageMixin:+ """+ SQLite 存储操作 Mixin++ 子类需要实现以下抽象方法:+ - _get_connection(date, db_type) -> sqlite3.Connection+ - _get_configured_time() -> datetime+ - _format_date_folder(date) -> str+ - _format_time_filename() -> str+ """++ # ========================================+ # 抽象方法 - 子类必须实现+ # ========================================++ @abstractmethod+ def _get_connection(self, date: Optional[str] = None, db_type: str = "news") -> sqlite3.Connection:+ """获取数据库连接"""+ pass++ @abstractmethod+ def _get_configured_time(self) -> datetime:+ """获取配置时区的当前时间"""+ pass++ @abstractmethod+ def _format_date_folder(self, date: Optional[str] = None) -> str:+ """格式化日期文件夹名 (ISO 格式: YYYY-MM-DD)"""+ pass++ @abstractmethod+ def _format_time_filename(self) -> str:+ """格式化时间文件名 (格式: HH-MM)"""+ pass++ # ========================================+ # Schema 管理+ # ========================================++ def _get_schema_path(self, db_type: str = "news") -> Path:+ """+ 获取 schema.sql 文件路径++ Args:+ db_type: 数据库类型 ("news" 或 "rss")++ Returns:+ schema 文件路径+ """+ if db_type == "rss":+ return Path(__file__).parent / "rss_schema.sql"+ return Path(__file__).parent / "schema.sql"++ def _init_tables(self, conn: sqlite3.Connection, db_type: str = "news") -> None:+ """+ 从 schema.sql 初始化数据库表结构++ Args:+ conn: 数据库连接+ db_type: 数据库类型 ("news" 或 "rss")+ """+ schema_path = self._get_schema_path(db_type)++ if schema_path.exists():+ with open(schema_path, "r", encoding="utf-8") as f:+ schema_sql = f.read()+ conn.executescript(schema_sql)+ else:+ raise FileNotFoundError(f"Schema file not found: {schema_path}")++ conn.commit()++ # ========================================+ # 新闻数据存储+ # ========================================++ def _save_news_data_impl(self, data: NewsData, log_prefix: str = "[存储]") -> tuple[bool, int, int, int, int]:+ """+ 保存新闻数据到 SQLite(核心实现)++ Args:+ data: 新闻数据+ log_prefix: 日志前缀++ Returns:+ (success, new_count, updated_count, title_changed_count, off_list_count)+ """+ try:+ conn = self._get_connection(data.date)+ cursor = conn.cursor()++ # 获取配置时区的当前时间+ now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")++ # 首先同步平台信息到 platforms 表+ for source_id, source_name in data.id_to_name.items():+ cursor.execute("""+ INSERT INTO platforms (id, name, updated_at)+ VALUES (?, ?, ?)+ ON CONFLICT(id) DO UPDATE SET+ name = excluded.name,+ updated_at = excluded.updated_at+ """, (source_id, source_name, now_str))++ # 统计计数器+ new_count = 0+ updated_count = 0+ title_changed_count = 0+ success_sources = []++ for source_id, news_list in data.items.items():+ success_sources.append(source_id)++ for item in news_list:+ try:+ # 标准化 URL(去除动态参数,如微博的 band_rank)+ normalized_url = normalize_url(item.url, source_id) if item.url else ""++ # 检查是否已存在(通过标准化 URL + platform_id)+ if normalized_url:+ cursor.execute("""+ SELECT id, title FROM news_items+ WHERE url = ? AND platform_id = ?+ """, (normalized_url, source_id))+ existing = cursor.fetchone()++ if existing:+ # 已存在,更新记录+ existing_id, existing_title = existing++ # 检查标题是否变化+ if existing_title != item.title:+ # 记录标题变更+ cursor.execute("""+ INSERT INTO title_changes+ (news_item_id, old_title, new_title, changed_at)+ VALUES (?, ?, ?, ?)+ """, (existing_id, existing_title, item.title, now_str))+ title_changed_count += 1++ # 记录排名历史+ cursor.execute("""+ INSERT INTO rank_history+ (news_item_id, rank, crawl_time, created_at)+ VALUES (?, ?, ?, ?)+ """, (existing_id, item.rank, data.crawl_time, now_str))++ # 更新现有记录+ cursor.execute("""+ UPDATE news_items SET+ title = ?,+ rank = ?,+ mobile_url = ?,+ last_crawl_time = ?,+ crawl_count = crawl_count + 1,+ updated_at = ?+ WHERE id = ?+ """, (item.title, item.rank, item.mobile_url,+ data.crawl_time, now_str, existing_id))+ updated_count += 1+ else:+ # 不存在,插入新记录(存储标准化后的 URL)+ cursor.execute("""+ INSERT INTO news_items+ (title, platform_id, rank, url, mobile_url,+ first_crawl_time, last_crawl_time, crawl_count,+ created_at, updated_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)+ """, (item.title, source_id, item.rank, normalized_url,+ item.mobile_url, data.crawl_time, data.crawl_time,+ now_str, now_str))+ new_id = cursor.lastrowid+ # 记录初始排名+ cursor.execute("""+ INSERT INTO rank_history+ (news_item_id, rank, crawl_time, created_at)+ VALUES (?, ?, ?, ?)+ """, (new_id, item.rank, data.crawl_time, now_str))+ new_count += 1+ else:+ # URL 为空的情况,直接插入(不做去重)+ cursor.execute("""+ INSERT INTO news_items+ (title, platform_id, rank, url, mobile_url,+ first_crawl_time, last_crawl_time, crawl_count,+ created_at, updated_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, 1, ?, ?)+ """, (item.title, source_id, item.rank, "",+ item.mobile_url, data.crawl_time, data.crawl_time,+ now_str, now_str))+ new_id = cursor.lastrowid+ # 记录初始排名+ cursor.execute("""+ INSERT INTO rank_history+ (news_item_id, rank, crawl_time, created_at)+ VALUES (?, ?, ?, ?)+ """, (new_id, item.rank, data.crawl_time, now_str))+ new_count += 1++ except sqlite3.Error as e:+ print(f"{log_prefix} 保存新闻条目失败 [{item.title[:30]}...]: {e}")++ total_items = new_count + updated_count++ # ========================================+ # 脱榜检测:检测上次在榜但这次不在榜的新闻+ # ========================================+ off_list_count = 0++ # 获取上一次抓取时间+ cursor.execute("""+ SELECT crawl_time FROM crawl_records+ WHERE crawl_time < ?+ ORDER BY crawl_time DESC+ LIMIT 1+ """, (data.crawl_time,))+ prev_record = cursor.fetchone()++ if prev_record:+ prev_crawl_time = prev_record[0]++ # 对于每个成功抓取的平台,检测脱榜+ for source_id in success_sources:+ # 获取当前抓取中该平台的所有标准化 URL+ current_urls = set()+ for item in data.items.get(source_id, []):+ normalized_url = normalize_url(item.url, source_id) if item.url else ""+ if normalized_url:+ current_urls.add(normalized_url)++ # 查询上次在榜(last_crawl_time = prev_crawl_time)但这次不在榜的新闻+ # 这些新闻是"第一次脱榜",需要记录+ cursor.execute("""+ SELECT id, url FROM news_items+ WHERE platform_id = ?+ AND last_crawl_time = ?+ AND url != ''+ """, (source_id, prev_crawl_time))++ for row in cursor.fetchall():+ news_id, url = row[0], row[1]+ if url not in current_urls:+ # 插入脱榜记录(rank=0 表示脱榜)+ cursor.execute("""+ INSERT INTO rank_history+ (news_item_id, rank, crawl_time, created_at)+ VALUES (?, 0, ?, ?)+ """, (news_id, data.crawl_time, now_str))+ off_list_count += 1++ # 记录抓取信息+ cursor.execute("""+ INSERT OR REPLACE INTO crawl_records+ (crawl_time, total_items, created_at)+ VALUES (?, ?, ?)+ """, (data.crawl_time, total_items, now_str))++ # 获取刚插入的 crawl_record 的 ID+ cursor.execute("""+ SELECT id FROM crawl_records WHERE crawl_time = ?+ """, (data.crawl_time,))+ record_row = cursor.fetchone()+ if record_row:+ crawl_record_id = record_row[0]++ # 记录成功的来源+ for source_id in success_sources:+ cursor.execute("""+ INSERT OR REPLACE INTO crawl_source_status+ (crawl_record_id, platform_id, status)+ VALUES (?, ?, 'success')+ """, (crawl_record_id, source_id))++ # 记录失败的来源+ for failed_id in data.failed_ids:+ # 确保失败的平台也在 platforms 表中+ cursor.execute("""+ INSERT OR IGNORE INTO platforms (id, name, updated_at)+ VALUES (?, ?, ?)+ """, (failed_id, failed_id, now_str))++ cursor.execute("""+ INSERT OR REPLACE INTO crawl_source_status+ (crawl_record_id, platform_id, status)+ VALUES (?, ?, 'failed')+ """, (crawl_record_id, failed_id))++ conn.commit()++ return True, new_count, updated_count, title_changed_count, off_list_count++ except Exception as e:+ print(f"{log_prefix} 保存失败: {e}")+ return False, 0, 0, 0, 0++ def _get_today_all_data_impl(self, date: Optional[str] = None) -> Optional[NewsData]:+ """+ 获取指定日期的所有新闻数据(合并后)++ Args:+ date: 日期字符串,默认为今天++ Returns:+ 合并后的新闻数据+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ # 获取所有新闻数据(包含 id 用于查询排名历史)+ cursor.execute("""+ SELECT n.id, n.title, n.platform_id, p.name as platform_name,+ n.rank, n.url, n.mobile_url,+ n.first_crawl_time, n.last_crawl_time, n.crawl_count+ FROM news_items n+ LEFT JOIN platforms p ON n.platform_id = p.id+ ORDER BY n.platform_id, n.last_crawl_time+ """)++ rows = cursor.fetchall()+ if not rows:+ return None++ # 收集所有 news_item_id+ news_ids = [row[0] for row in rows]++ # 批量查询排名历史(同时获取时间和排名)+ # 过滤逻辑:只保留 last_crawl_time 之前的脱榜记录(rank=0)+ # 这样可以避免显示新闻永久脱榜后的无意义记录+ rank_history_map: Dict[int, List[int]] = {}+ rank_timeline_map: Dict[int, List[Dict[str, Any]]] = {}+ if news_ids:+ placeholders = ",".join("?" * len(news_ids))+ cursor.execute(f"""+ SELECT rh.news_item_id, rh.rank, rh.crawl_time+ FROM rank_history rh+ JOIN news_items ni ON rh.news_item_id = ni.id+ WHERE rh.news_item_id IN ({placeholders})+ AND NOT (rh.rank = 0 AND rh.crawl_time > ni.last_crawl_time)+ ORDER BY rh.news_item_id, rh.crawl_time+ """, news_ids)+ for rh_row in cursor.fetchall():+ news_id, rank, crawl_time = rh_row[0], rh_row[1], rh_row[2]++ # 构建 ranks 列表(去重,排除脱榜记录 rank=0)+ if news_id not in rank_history_map:+ rank_history_map[news_id] = []+ if rank != 0 and rank not in rank_history_map[news_id]:+ rank_history_map[news_id].append(rank)++ # 构建 rank_timeline 列表(完整时间线,包含脱榜)+ if news_id not in rank_timeline_map:+ rank_timeline_map[news_id] = []+ # 提取时间部分(HH:MM)+ time_part = crawl_time.split()[1][:5] if ' ' in crawl_time else crawl_time[:5]+ rank_timeline_map[news_id].append({+ "time": time_part,+ "rank": rank if rank != 0 else None # 0 转为 None 表示脱榜+ })++ # 按 platform_id 分组+ items: Dict[str, List[NewsItem]] = {}+ id_to_name: Dict[str, str] = {}+ crawl_date = self._format_date_folder(date)++ for row in rows:+ news_id = row[0]+ platform_id = row[2]+ title = row[1]+ platform_name = row[3] or platform_id++ id_to_name[platform_id] = platform_name++ if platform_id not in items:+ items[platform_id] = []++ # 获取排名历史,如果没有则使用当前排名+ ranks = rank_history_map.get(news_id, [row[4]])+ rank_timeline = rank_timeline_map.get(news_id, [])++ items[platform_id].append(NewsItem(+ title=title,+ source_id=platform_id,+ source_name=platform_name,+ rank=row[4],+ url=row[5] or "",+ mobile_url=row[6] or "",+ crawl_time=row[8], # last_crawl_time+ ranks=ranks,+ first_time=row[7], # first_crawl_time+ last_time=row[8], # last_crawl_time+ count=row[9], # crawl_count+ rank_timeline=rank_timeline,+ ))++ final_items = items++ # 获取失败的来源+ cursor.execute("""+ SELECT DISTINCT css.platform_id+ FROM crawl_source_status css+ JOIN crawl_records cr ON css.crawl_record_id = cr.id+ WHERE css.status = 'failed'+ """)+ failed_ids = [row[0] for row in cursor.fetchall()]++ # 获取最新的抓取时间+ cursor.execute("""+ SELECT crawl_time FROM crawl_records+ ORDER BY crawl_time DESC+ LIMIT 1+ """)++ time_row = cursor.fetchone()+ crawl_time = time_row[0] if time_row else self._format_time_filename()++ return NewsData(+ date=crawl_date,+ crawl_time=crawl_time,+ items=final_items,+ id_to_name=id_to_name,+ failed_ids=failed_ids,+ )++ except Exception as e:+ print(f"[存储] 读取数据失败: {e}")+ return None++ def _get_latest_crawl_data_impl(self, date: Optional[str] = None) -> Optional[NewsData]:+ """+ 获取最新一次抓取的数据++ Args:+ date: 日期字符串,默认为今天++ Returns:+ 最新抓取的新闻数据+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ # 获取最新的抓取时间+ cursor.execute("""+ SELECT crawl_time FROM crawl_records+ ORDER BY crawl_time DESC+ LIMIT 1+ """)++ time_row = cursor.fetchone()+ if not time_row:+ return None++ latest_time = time_row[0]++ # 获取该时间的新闻数据(包含 id 用于查询排名历史)+ cursor.execute("""+ SELECT n.id, n.title, n.platform_id, p.name as platform_name,+ n.rank, n.url, n.mobile_url,+ n.first_crawl_time, n.last_crawl_time, n.crawl_count+ FROM news_items n+ LEFT JOIN platforms p ON n.platform_id = p.id+ WHERE n.last_crawl_time = ?+ """, (latest_time,))++ rows = cursor.fetchall()+ if not rows:+ return None++ # 收集所有 news_item_id+ news_ids = [row[0] for row in rows]++ # 批量查询排名历史(同时获取时间和排名)+ # 过滤逻辑:只保留 last_crawl_time 之前的脱榜记录(rank=0)+ # 这样可以避免显示新闻永久脱榜后的无意义记录+ rank_history_map: Dict[int, List[int]] = {}+ rank_timeline_map: Dict[int, List[Dict[str, Any]]] = {}+ if news_ids:+ placeholders = ",".join("?" * len(news_ids))+ cursor.execute(f"""+ SELECT rh.news_item_id, rh.rank, rh.crawl_time+ FROM rank_history rh+ JOIN news_items ni ON rh.news_item_id = ni.id+ WHERE rh.news_item_id IN ({placeholders})+ AND NOT (rh.rank = 0 AND rh.crawl_time > ni.last_crawl_time)+ ORDER BY rh.news_item_id, rh.crawl_time+ """, news_ids)+ for rh_row in cursor.fetchall():+ news_id, rank, crawl_time = rh_row[0], rh_row[1], rh_row[2]++ # 构建 ranks 列表(去重,排除脱榜记录 rank=0)+ if news_id not in rank_history_map:+ rank_history_map[news_id] = []+ if rank != 0 and rank not in rank_history_map[news_id]:+ rank_history_map[news_id].append(rank)++ # 构建 rank_timeline 列表(完整时间线,包含脱榜)+ if news_id not in rank_timeline_map:+ rank_timeline_map[news_id] = []+ # 提取时间部分(HH:MM)+ time_part = crawl_time.split()[1][:5] if ' ' in crawl_time else crawl_time[:5]+ rank_timeline_map[news_id].append({+ "time": time_part,+ "rank": rank if rank != 0 else None # 0 转为 None 表示脱榜+ })++ items: Dict[str, List[NewsItem]] = {}+ id_to_name: Dict[str, str] = {}+ crawl_date = self._format_date_folder(date)++ for row in rows:+ news_id = row[0]+ platform_id = row[2]+ platform_name = row[3] or platform_id+ id_to_name[platform_id] = platform_name++ if platform_id not in items:+ items[platform_id] = []++ # 获取排名历史,如果没有则使用当前排名+ ranks = rank_history_map.get(news_id, [row[4]])+ rank_timeline = rank_timeline_map.get(news_id, [])++ items[platform_id].append(NewsItem(+ title=row[1],+ source_id=platform_id,+ source_name=platform_name,+ rank=row[4],+ url=row[5] or "",+ mobile_url=row[6] or "",+ crawl_time=row[8], # last_crawl_time+ ranks=ranks,+ first_time=row[7], # first_crawl_time+ last_time=row[8], # last_crawl_time+ count=row[9], # crawl_count+ rank_timeline=rank_timeline,+ ))++ # 获取失败的来源(针对最新一次抓取)+ cursor.execute("""+ SELECT css.platform_id+ FROM crawl_source_status css+ JOIN crawl_records cr ON css.crawl_record_id = cr.id+ WHERE cr.crawl_time = ? AND css.status = 'failed'+ """, (latest_time,))++ failed_ids = [row[0] for row in cursor.fetchall()]++ return NewsData(+ date=crawl_date,+ crawl_time=latest_time,+ items=items,+ id_to_name=id_to_name,+ failed_ids=failed_ids,+ )++ except Exception as e:+ print(f"[存储] 获取最新数据失败: {e}")+ return None++ def _detect_new_titles_impl(self, current_data: NewsData) -> Dict[str, Dict]:+ """+ 检测新增的标题++ 该方法比较当前抓取数据与历史数据,找出新增的标题。+ 关键逻辑:只有在历史批次中从未出现过的标题才算新增。++ Args:+ current_data: 当前抓取的数据++ Returns:+ 新增的标题数据 {source_id: {title: NewsItem}}+ """+ try:+ # 获取历史数据+ historical_data = self._get_today_all_data_impl(current_data.date)++ if not historical_data:+ # 没有历史数据,所有都是新的+ new_titles = {}+ for source_id, news_list in current_data.items.items():+ new_titles[source_id] = {item.title: item for item in news_list}+ return new_titles++ # 获取当前批次时间+ current_time = current_data.crawl_time++ # 收集历史标题(first_time < current_time 的标题)+ # 这样可以正确处理同一标题因 URL 变化而产生多条记录的情况+ historical_titles: Dict[str, set] = {}+ for source_id, news_list in historical_data.items.items():+ historical_titles[source_id] = set()+ for item in news_list:+ first_time = getattr(item, 'first_time', item.crawl_time)+ if first_time < current_time:+ historical_titles[source_id].add(item.title)++ # 检查是否有历史数据+ has_historical_data = any(len(titles) > 0 for titles in historical_titles.values())+ if not has_historical_data:+ # 第一次抓取,没有"新增"概念+ return {}++ # 检测新增+ new_titles = {}+ for source_id, news_list in current_data.items.items():+ hist_set = historical_titles.get(source_id, set())+ for item in news_list:+ if item.title not in hist_set:+ if source_id not in new_titles:+ new_titles[source_id] = {}+ new_titles[source_id][item.title] = item++ return new_titles++ except Exception as e:+ print(f"[存储] 检测新标题失败: {e}")+ return {}++ def _is_first_crawl_today_impl(self, date: Optional[str] = None) -> bool:+ """+ 检查是否是当天第一次抓取++ Args:+ date: 日期字符串,默认为今天++ Returns:+ 是否是第一次抓取+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ cursor.execute("""+ SELECT COUNT(*) as count FROM crawl_records+ """)++ row = cursor.fetchone()+ count = row[0] if row else 0++ # 如果只有一条或没有记录,视为第一次抓取+ return count <= 1++ except Exception as e:+ print(f"[存储] 检查首次抓取失败: {e}")+ return True++ def _get_crawl_times_impl(self, date: Optional[str] = None) -> List[str]:+ """+ 获取指定日期的所有抓取时间列表++ Args:+ date: 日期字符串,默认为今天++ Returns:+ 抓取时间列表(按时间排序)+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ cursor.execute("""+ SELECT crawl_time FROM crawl_records+ ORDER BY crawl_time+ """)++ rows = cursor.fetchall()+ return [row[0] for row in rows]++ except Exception as e:+ print(f"[存储] 获取抓取时间列表失败: {e}")+ return []++ # ========================================+ # 推送记录+ # ========================================++ def _has_pushed_today_impl(self, date: Optional[str] = None) -> bool:+ """+ 检查指定日期是否已推送过++ Args:+ date: 日期字符串(YYYY-MM-DD),默认为今天++ Returns:+ 是否已推送+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ target_date = self._format_date_folder(date)++ cursor.execute("""+ SELECT pushed FROM push_records WHERE date = ?+ """, (target_date,))++ row = cursor.fetchone()+ if row:+ return bool(row[0])+ return False++ except Exception as e:+ print(f"[存储] 检查推送记录失败: {e}")+ return False++ def _record_push_impl(self, report_type: str, date: Optional[str] = None) -> bool:+ """+ 记录推送++ Args:+ report_type: 报告类型+ date: 日期字符串(YYYY-MM-DD),默认为今天++ Returns:+ 是否记录成功+ """+ try:+ conn = self._get_connection(date)+ cursor = conn.cursor()++ target_date = self._format_date_folder(date)+ now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")++ cursor.execute("""+ INSERT INTO push_records (date, pushed, push_time, report_type, created_at)+ VALUES (?, 1, ?, ?, ?)+ ON CONFLICT(date) DO UPDATE SET+ pushed = 1,+ push_time = excluded.push_time,+ report_type = excluded.report_type+ """, (target_date, now_str, report_type, now_str))++ conn.commit()+ return True++ except Exception as e:+ print(f"[存储] 记录推送失败: {e}")+ return False++ # ========================================+ # RSS 数据存储+ # ========================================++ def _save_rss_data_impl(self, data: RSSData, log_prefix: str = "[存储]") -> tuple[bool, int, int]:+ """+ 保存 RSS 数据到 SQLite(以 URL 为唯一标识)++ Args:+ data: RSS 数据+ log_prefix: 日志前缀++ Returns:+ (success, new_count, updated_count)+ """+ try:+ conn = self._get_connection(data.date, db_type="rss")+ cursor = conn.cursor()++ now_str = self._get_configured_time().strftime("%Y-%m-%d %H:%M:%S")++ # 同步 RSS 源信息到 rss_feeds 表+ for feed_id, feed_name in data.id_to_name.items():+ cursor.execute("""+ INSERT INTO rss_feeds (id, name, updated_at)+ VALUES (?, ?, ?)+ ON CONFLICT(id) DO UPDATE SET+ name = excluded.name,+ updated_at = excluded.updated_at+ """, (feed_id, feed_name, now_str))++ # 统计计数器+ new_count = 0+ updated_count = 0++ for feed_id, rss_list in data.items.items():+ for item in rss_list:+ try:+ # 检查是否已存在(通过 URL + feed_id)+ if item.url:+ cursor.execute("""+ SELECT id, title FROM rss_items+ WHERE url = ? AND feed_id = ?+ """, (item.url, feed_id))+ existing = cursor.fetchone()++ if existing:+ # 已存在,更新记录+ existing_id = existing[0]+ cursor.execute("""+ UPDATE rss_items SET+ title = ?,+ published_at = ?,+ summary = ?,+ author = ?,+ last_crawl_time = ?,+ crawl_count = crawl_count + 1,+ updated_at = ?+ WHERE id = ?+ """, (item.title, item.published_at, item.summary,+ item.author, data.crawl_time, now_str, existing_id))+ updated_count += 1+ else:+ # 不存在,插入新记录(使用 ON CONFLICT 兜底处理并发/竞争场景)+ cursor.execute("""+ INSERT INTO rss_items+ (title, feed_id, url, published_at, summary, author,+ first_crawl_time, last_crawl_time, crawl_count,+ created_at, updated_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)+ ON CONFLICT(url, feed_id) DO UPDATE SET+ title = excluded.title,+ published_at = excluded.published_at,+ summary = excluded.summary,+ author = excluded.author,+ last_crawl_time = excluded.last_crawl_time,+ crawl_count = crawl_count + 1,+ updated_at = excluded.updated_at+ """, (item.title, feed_id, item.url, item.published_at,+ item.summary, item.author, data.crawl_time,+ data.crawl_time, now_str, now_str))+ new_count += 1+ else:+ # URL 为空,用 try-except 处理重复+ try:+ cursor.execute("""+ INSERT INTO rss_items+ (title, feed_id, url, published_at, summary, author,+ first_crawl_time, last_crawl_time, crawl_count,+ created_at, updated_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, 1, ?, ?)+ """, (item.title, feed_id, "", item.published_at,+ item.summary, item.author, data.crawl_time,+ data.crawl_time, now_str, now_str))+ new_count += 1+ except sqlite3.IntegrityError:+ # 重复的空 URL 条目,忽略+ pass++ except sqlite3.Error as e:+ print(f"{log_prefix} 保存 RSS 条目失败 [{item.title[:30]}...]: {e}")++ total_items = new_count + updated_count++ # 记录抓取信息+ cursor.execute("""+ INSERT OR REPLACE INTO rss_crawl_records+ (crawl_time, total_items, created_at)+ VALUES (?, ?, ?)+ """, (data.crawl_time, total_items, now_str))++ # 记录抓取状态+ cursor.execute("""+ SELECT id FROM rss_crawl_records WHERE crawl_time = ?+ """, (data.crawl_time,))+ record_row = cursor.fetchone()+ if record_row:+ crawl_record_id = record_row[0]++ # 记录成功的源+ for feed_id in data.items.keys():+ cursor.execute("""+ INSERT OR REPLACE INTO rss_crawl_status+ (crawl_record_id, feed_id, status)+ VALUES (?, ?, 'success')+ """, (crawl_record_id, feed_id))++ # 记录失败的源+ for failed_id in data.failed_ids:+ cursor.execute("""+ INSERT OR IGNORE INTO rss_feeds (id, name, updated_at)+ VALUES (?, ?, ?)+ """, (failed_id, failed_id, now_str))++ cursor.execute("""+ INSERT OR REPLACE INTO rss_crawl_status+ (crawl_record_id, feed_id, status)+ VALUES (?, ?, 'failed')+ """, (crawl_record_id, failed_id))++ conn.commit()++ return True, new_count, updated_count++ except Exception as e:+ print(f"{log_prefix} 保存 RSS 数据失败: {e}")+ return False, 0, 0++ def _get_rss_data_impl(self, date: Optional[str] = None) -> Optional[RSSData]:+ """+ 获取指定日期的所有 RSS 数据++ Args:+ date: 日期字符串(YYYY-MM-DD),默认为今天++ Returns:+ RSSData 对象,如果没有数据返回 None+ """+ try:+ conn = self._get_connection(date, db_type="rss")+ cursor = conn.cursor()++ # 获取所有 RSS 数据+ cursor.execute("""+ SELECT i.id, i.title, i.feed_id, f.name as feed_name,+ i.url, i.published_at, i.summary, i.author,+ i.first_crawl_time, i.last_crawl_time, i.crawl_count+ FROM rss_items i+ LEFT JOIN rss_feeds f ON i.feed_id = f.id+ ORDER BY i.published_at DESC+ """)++ rows = cursor.fetchall()+ if not rows:+ return None++ items: Dict[str, List[RSSItem]] = {}+ id_to_name: Dict[str, str] = {}+ crawl_date = self._format_date_folder(date)++ for row in rows:+ feed_id = row[2]+ feed_name = row[3] or feed_id++ id_to_name[feed_id] = feed_name++ if feed_id not in items:+ items[feed_id] = []++ items[feed_id].append(RSSItem(+ title=row[1],+ feed_id=feed_id,+ feed_name=feed_name,+ url=row[4] or "",+ published_at=row[5] or "",+ summary=row[6] or "",+ author=row[7] or "",+ crawl_time=row[9],+ first_time=row[8],+ last_time=row[9],+ count=row[10],+ ))++ # 获取最新的抓取时间+ cursor.execute("""+ SELECT crawl_time FROM rss_crawl_records+ ORDER BY crawl_time DESC+ LIMIT 1+ """)+ time_row = cursor.fetchone()+ crawl_time = time_row[0] if time_row else self._format_time_filename()++ # 获取失败的源+ cursor.execute("""+ SELECT DISTINCT cs.feed_id+ FROM rss_crawl_status cs+ JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id+ WHERE cs.status = 'failed'+ """)+ failed_ids = [row[0] for row in cursor.fetchall()]++ return RSSData(+ date=crawl_date,+ crawl_time=crawl_time,+ items=items,+ id_to_name=id_to_name,+ failed_ids=failed_ids,+ )++ except Exception as e:+ print(f"[存储] 读取 RSS 数据失败: {e}")+ return None++ def _detect_new_rss_items_impl(self, current_data: RSSData) -> Dict[str, List[RSSItem]]:+ """+ 检测新增的 RSS 条目(增量模式)++ 该方法比较当前抓取数据与历史数据,找出新增的 RSS 条目。+ 关键逻辑:只有在历史批次中从未出现过的 URL 才算新增。++ Args:+ current_data: 当前抓取的 RSS 数据++ Returns:+ 新增的 RSS 条目 {feed_id: [RSSItem, ...]}+ """+ try:+ # 获取历史数据+ historical_data = self._get_rss_data_impl(current_data.date)++ if not historical_data:+ # 没有历史数据,所有都是新的+ return current_data.items.copy()++ # 获取当前批次时间+ current_time = current_data.crawl_time++ # 收集历史 URL(first_time < current_time 的条目)+ historical_urls: Dict[str, set] = {}+ for feed_id, rss_list in historical_data.items.items():+ historical_urls[feed_id] = set()+ for item in rss_list:+ first_time = getattr(item, 'first_time', item.crawl_time)+ if first_time < current_time:+ if item.url:+ historical_urls[feed_id].add(item.url)++ # 检查是否有历史数据+ has_historical_data = any(len(urls) > 0 for urls in historical_urls.values())+ if not has_historical_data:+ # 第一次抓取,没有"新增"概念+ return {}++ # 检测新增+ new_items: Dict[str, List[RSSItem]] = {}+ for feed_id, rss_list in current_data.items.items():+ hist_set = historical_urls.get(feed_id, set())+ for item in rss_list:+ # 通过 URL 判断是否新增+ if item.url and item.url not in hist_set:+ if feed_id not in new_items:+ new_items[feed_id] = []+ new_items[feed_id].append(item)++ return new_items++ except Exception as e:+ print(f"[存储] 检测新 RSS 条目失败: {e}")+ return {}++ def _get_latest_rss_data_impl(self, date: Optional[str] = None) -> Optional[RSSData]:+ """+ 获取最新一次抓取的 RSS 数据(当前榜单模式)++ Args:+ date: 日期字符串(YYYY-MM-DD),默认为今天++ Returns:+ 最新抓取的 RSS 数据,如果没有数据返回 None+ """+ try:+ conn = self._get_connection(date, db_type="rss")+ cursor = conn.cursor()++ # 获取最新的抓取时间+ cursor.execute("""+ SELECT crawl_time FROM rss_crawl_records+ ORDER BY crawl_time DESC+ LIMIT 1+ """)++ time_row = cursor.fetchone()+ if not time_row:+ return None++ latest_time = time_row[0]++ # 获取该时间的 RSS 数据+ cursor.execute("""+ SELECT i.id, i.title, i.feed_id, f.name as feed_name,+ i.url, i.published_at, i.summary, i.author,+ i.first_crawl_time, i.last_crawl_time, i.crawl_count+ FROM rss_items i+ LEFT JOIN rss_feeds f ON i.feed_id = f.id+ WHERE i.last_crawl_time = ?+ ORDER BY i.published_at DESC+ """, (latest_time,))++ rows = cursor.fetchall()+ if not rows:+ return None++ items: Dict[str, List[RSSItem]] = {}+ id_to_name: Dict[str, str] = {}+ crawl_date = self._format_date_folder(date)++ for row in rows:+ feed_id = row[2]+ feed_name = row[3] or feed_id++ id_to_name[feed_id] = feed_name++ if feed_id not in items:+ items[feed_id] = []++ items[feed_id].append(RSSItem(+ title=row[1],+ feed_id=feed_id,+ feed_name=feed_name,+ url=row[4] or "",+ published_at=row[5] or "",+ summary=row[6] or "",+ author=row[7] or "",+ crawl_time=row[9],+ first_time=row[8],+ last_time=row[9],+ count=row[10],+ ))++ # 获取失败的源(针对最新一次抓取)+ cursor.execute("""+ SELECT cs.feed_id+ FROM rss_crawl_status cs+ JOIN rss_crawl_records cr ON cs.crawl_record_id = cr.id+ WHERE cr.crawl_time = ? AND cs.status = 'failed'+ """, (latest_time,))++ failed_ids = [row[0] for row in cursor.fetchall()]++ return RSSData(+ date=crawl_date,+ crawl_time=latest_time,+ items=items,+ id_to_name=id_to_name,+ failed_ids=failed_ids,+ )++ except Exception as e:+ print(f"[存储] 获取最新 RSS 数据失败: {e}")+ return Nonediff --git a/version b/versionindex 28cbf7c0aa..7cbea073be 100644--- a/version+++ b/version@@ -1 +1 @@-5.0.0\ No newline at end of file+5.2.0\ No newline at end of filediff --git a/version_mcp b/version_mcpindex 97ceee1b9f..8a4b2758ef 100644--- a/version_mcp+++ b/version_mcp@@ -1 +1 @@-3.1.5\ No newline at end of file+3.1.6\ No newline at end of file