Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
更新 README 文档,优化 Hyper-Extract 的描述,增强对领域知识模板的介绍,调整金融领域模板的内容和结构
yifanfeng975ad88164ab9f
diff --git a/README.md b/README.mdindex 8dfb8c5..7c3c60e 100644--- a/README.md+++ b/README.md@@ -1,83 +1,77 @@-# Hyper-Extract: From Text to Actionable Knowledge Structures 🚀+# Hyper-Extract: The Knowledge Engine for Vertical AI Agents 🧠🚀[](LICENSE)[](pyproject.toml)-**Hyper-Extract** is a next-generation knowledge extraction framework designed to transform unstructured text into **searchable, queryable, and actionable** structured knowledge.+**Hyper-Extract** is a next-generation framework designed to transform massive, unstructured domain documents into **searchable, queryable, and actionable** knowledge structures.-Unlike traditional extraction tools that only output static data, Hyper-Extract automatically converts extracted structures (Lists, Graphs, Hypergraphs) into an interactive knowledge base. You can directly chat with the structure, perform precise semantic searches, and realize "Extraction as an Application."+Stop treating your documents as dead text chunks. With Hyper-Extract, every PDF, report, and manual becomes a live, structured brain for your AI Agents.-[中文版自述文件 (README_ZH.md)](README_ZH.md)+[中文版 (README_ZH.md)](README_ZH.md) | [Domain Samples](examples/)----## 🌟 Key Highlights+## 🌟 Why Hyper-Extract?-- **📂 Universal Structure Support**:- - **Foundational**: List, Set, General Knowledge Graph.- - **Dynamics**: Temporal Graph (time-series events), Spatial Graph (geographical topology), Spatio-Temporal Graph.- - **Complex Relations**: Hypergraph (N-ary relations), perfect for modeling group interactions and holistic context.-- **💬 Actionable Knowledge (RAG 2.0)**:- - Extraction is just the beginning. Hyper-Extract automatically builds vector indices for every extracted entity and relationship.- - **Chat with Structure**: Query your graph or list directly to get answers backed by structured logic.-- **🛠️ Production-Ready SOTA Algorithms**:- - Built-in implementations of cutting-edge extraction and RAG paradigms: `Auto-Gen`, `Hyper-RAG`, `Light-RAG`, `GraphRAG`, `CogRAG`, and more.-- **🎭 Massive Domain Templates**:- - 20+ out-of-the-box templates covering: Finance, Medicine, Law, Gaming, Agriculture, Cybersecurity, Physics, Literature, etc.+Traditional RAG often fails in professional domains because it loses the **logical structure** of knowledge. Hyper-Extract solves this by providing a unique **3-Layer Architecture**:++### 🧱 Layer 1: Knowledge Primitives (`hyperextract.types`)+The "DNA" of knowledge. We provide specialized "AutoType" structures categorized into four levels of complexity:++| Level | Primitives | Core Logic | Best For |+| :--- | :--- | :--- | :--- |+| **Scalar & Doc** | `AutoModel` | Many chunks -> **One** consistent object | Research reports, Infoboxes, Summaries |+| **Collections** | `AutoList`, `AutoSet` | Many independent or unique items | Entity registries, Event logs, Glossaries |+| **Graphs** | `AutoGraph`, `AutoHypergraph` | Pairwise or N-ary (Hyper-edge) relations | Knowledge Graphs, Complex causality reasoning |+| **Context-Aware** | `AutoTemporalGraph`, `AutoSpatialGraph`, `AutoSpatioTemporalGraph` | Resolves relative time/space context | Timelines, History, IoT |++### ⚙️ Layer 2: Extraction Engines (`hyperextract.methods`)+The "Brain" of the framework. Built-in SOTA algorithms that turn primitives into power:+- **Light-RAG / GraphRAG**: High-efficiency retrieval over graph structures.+- **Hyper-RAG**: Reasoning via complex hyper-relations.+- **CogRAG**: Cognition-driven hierarchical extraction.++### 🧠 Layer 3: 12+ Domain Expert Templates (`hyperextract.templates`)+The "Expertise" you need. Out-of-the-box schemas and prompts for vertical industries:+- **Finance, Medicine, Law, TCM, Industry, History, Biology, Literature, Travel, News, Agriculture, Food.**----## 🚀 Quick Start+## 🎭 Transform Your Domain in Minutes-### Installation+"From static files to an intelligent agent engine."-```bash-pip install hyperextract-```+| Your Domain | From Static Docs ... | To Actionable Agents 🤖 |+| :--- | :--- | :--- |+| **💰 Finance** | Annual reports, prospectuses | **Investment Analyst Agent** (Reasoning over valuation logic) |+| **🏥 Medicine** | Clinical guidelines, patient records | **Diagnostic Assistant Agent** (Mapping symptoms to pathways) |+| **⚖️ Legal** | Contracts, court rulings | **Compliance Auditor Agent** (Extracting obligations & risks) |+| **🔧 Industry** | Maintenance logs, specs | **Ops Expert Agent** (Connecting failure modes to root causes) |+| **📜 History** | Chronicles, biographies | **Historian Agent** (Reconstructing event timelines) |-### Build an "Intelligent Knowledge Graph" in 3 Steps+----Hyper-Extract makes complex extraction and interaction dead simple:+## 🚀 Quick Start: Build an "Askable" Knowledge Base```python-from pydantic import BaseModel, Fieldfrom hyperextract.types import AutoGraph-from langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.templates.en.finance import ResearchNoteSummary+from langchain_openai import ChatOpenAI-# 1. Define your world view (Schema)-class entity(BaseModel):- name: str = Field(description="Name of the entity")- type: str = Field(description="Category, e.g., Person, Company, Location")+# 1. Load an industry-expert template+llm = ChatOpenAI(model="gpt-4o")+kb = AutoGraph.from_template(ResearchNoteSummary, llm_client=llm)-class edge(BaseModel):- source: str = Field(description="Source entity name")- target: str = Field(description="Target entity name")- relation: str = Field(description="The relationship between them")+# 2. Feed your domain text (Extraction happens automatically!)+text = "Apple's valuation is driven by its services growth and iPhone 16 cycle..."+kb.feed_text(text)-# 2. Initialize and Build from Text-llm = ChatOpenAI(model="gpt-4o")-embedder = OpenAIEmbeddings()--graph_kb = AutoGraph(- node_schema=entity,- edge_schema=edge,- # Unique key extractors for deduplication and linking- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}_{x.relation}_{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder-)--text = "Apple Inc. was founded by Steve Jobs in California, and it is the maker of the iPhone."-graph_kb.feed_text(text) # Extract knowledge-graph_kb.build_index() # Build vector index for Q&A--# 3. Chat with the Structure (Actionable!)-answer = graph_kb.chat("What is the relationship between Jobs and Apple?")+# 3. Chat with the Structure (Actionable Insight)+answer = kb.chat("What are the core drivers of Apple's valuation?")print(answer.content)```+---## 🎨 Domain Templatesdiff --git a/README_ZH.md b/README_ZH.mdindex cfc5c0d..10e7889 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -1,83 +1,77 @@-# Hyper-Extract: 从文本到可交互的知识结构 🚀+# Hyper-Extract: 垂直领域 AI Agent 的知识引擎 🧠🚀[](LICENSE)[](pyproject.toml)-**Hyper-Extract** 是一个下一代知识提取框架,旨在将非结构化文本转化为**可搜索、可查询、可问答**的结构化知识。+**Hyper-Extract** 是一个下一代知识提取框架,旨在将海量、非结构化的领域文档转化为**可搜索、可查询、可推理**的知识结构。-区别于传统的只负责“提取”的工具,Hyper-Extract 将提取出的结构(如列表、图谱、超图)自动转化为可交互的知识库。你可以直接与提取出的结构进行对话,进行精确的语义搜索,实现“提取即应用”。+不要再把你的文档当作死气沉沉的文本切片。通过 Hyper-Extract,每一份 PDF、研报、说明书都能转化为 AI Agent 动态调用的“活体大脑”。-[English README](README.md)+[English (README.md)](README.md) | [示例代码](examples/)----## 🌟 核心特性+## 🌟 为什么选择 Hyper-Extract?-- **📂 全模态结构支持**:- - **基础结构**: List (列表), Set (集合), Graph (通用图谱)。- - **时空动力学**: Temporal Graph (时序图), Spatial Graph (地理图), Spatio-Temporal Graph (时空图)。- - **复杂关联**: Hypergraph (超图/多元图),支持多对多复杂关系建模。-- **💬 结构化知识问答 (RAG 2.0)**:- - 提取不仅仅是终点。Hyper-Extract 自动为提取出的实体和关系建立向量索引。- - **Chat with Structure**: 直接针对图谱或集合进行问答,获得具有结构化逻辑支撑的答案。-- **🛠️ 工业级算法复现**:- - 内置了多种 SOTA 提取与检索算法的复现,包括 `Auto-Gen`, `Hyper-RAG`, `Light-RAG`, `GraphRAG`, `CogRAG` 等。-- **🎭 领域专家模版库**:- - 提供了 20+ 个领域的开箱即用模版,涵盖:金融、医疗、法律、游戏、农业、网络安全、物理、文学等。+传统的 RAG 模型在垂直和专业领域往往因为丢失知识的**逻辑结构**而失效。Hyper-Extract 通过独特的**三层分级架构**解决这一痛点:++### 🧱 第一级:知识基础原语 (`hyperextract.types`)+知识的 "DNA"。我们根据信息复杂度的不同,将 "AutoType" 结构分为四个层级:++| 层级 | 基础原语 | 核心逻辑 | 适用场景 |+| :--- | :--- | :--- | :--- |+| **标量与文档型** | `AutoModel` | 多个切片 -> **一个**一致性的对象 | 研报摘要、百科条目、元数据提取 |+| **集合型** | `AutoList`, `AutoSet` | 独立项或唯一实体的去重集合 | 实体清单、术语表、RPG 手册 |+| **图谱型** | `AutoGraph`, `AutoHypergraph` | 二元 (Pairwise) 或多元 (N-ary) 关系 | 知识图谱、多因素复杂因果推理 |+| **时空感知型** | `AutoTemporalGraph`, `AutoSpatialGraph`, `AutoSpatioTemporalGraph` | 解析相对的时间/空间上下文 | 编年史、时政新闻、拓扑描述 |++### ⚙️ 第二级:核心提取引擎 (`hyperextract.methods`)+系统的 "大脑"。内置 SOTA 算法,让数据结构产生生产力:+- **Light-RAG / GraphRAG**: 在图谱结构上进行高效率的全局检索。+- **Hyper-RAG / CogRAG**: 基于复杂超图和认知层级的推理问答引擎。+- **Typical Methods**: 支持 iText2KG, KG-Gen 等经典提取逻辑。++### 🧠 第三级:12+ 行业领域专家模板 (`hyperextract.templates`)+你的专业 "灵魂"。开箱即用的行业 Schema 和 Prompt:+- **金融、医疗、法律、中医、工业、历史、生物、文学、旅游、新闻、农业、美食。**----## 🚀 快速上手+## 🎭 瞬间转化:从静态文档到行业专家 Agent 🤖-### 安装+"让每一行文本都具有交互性。"-```bash-pip install hyperextract-```+| 领域 | 原始文档输入 | 转化后的 Agent 能力 |+| :--- | :--- | :--- |+| **💰 金融** | 研报、公告、招股书 | **投研助手 Agent** (推理各公司估值逻辑与驱动因素) |+| **🏥 医疗** | 临床指南、病案、说明书 | **诊疗分析 Agent** (从症状推演鉴别诊断路径) |+| **⚖️ 法律** | 合同协议、判决文书 | **合规风控 Agent** (一分钟内提取复杂责任与违约关系) |+| **🔧 工业** | 运维日志、技术规格书 | **排故专家 Agent** (关联故障现象与其根本原因) |+| **📜 历史** | 编年史、信札、传记 | **文史学家 Agent** (还原复杂的历史事件因果链条) |-### 3步实现“对话知识图谱”+----Hyper-Extract 让复杂结构的提取和使用变得异常简单:+## 🚀 快速上手:构建一个“可对话”的领域知识库```python-from pydantic import BaseModel, Fieldfrom hyperextract.types import AutoGraph-from langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.templates.zh.finance import ResearchNoteSummary+from langchain_openai import ChatOpenAI-# 1. 定义你想要提取的结构 (Schema)-class entity(BaseModel):- name: str = Field(description="实体名称")- type: str = Field(description="类别,如:人、公司、地点")+# 1. 加载一个行业模板 (例如: 金融研报摘要)+llm = ChatOpenAI(model="gpt-4o")+kb = AutoGraph.from_template(ResearchNoteSummary, llm_client=llm)-class edge(BaseModel):- source: str = Field(description="源实体")- target: str = Field(description="目标实体")- relation: str = Field(description="它们之间的关系")+# 2. “喂入” 领域文本 (自动完成知识提取与建模)+text = "苹果公司的估值核心逻辑在于其服务收入的增长以及 iPhone 16 的换机周期..."+kb.feed_text(text)-# 2. 初始化提取器并从文本构建图谱-llm = ChatOpenAI(model="gpt-4o")-embedder = OpenAIEmbeddings()--graph_kb = AutoGraph(- node_schema=entity,- edge_schema=edge,- # 唯一键提取规则,用于去重和关联- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}_{x.relation}_{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder-)--text = "苹果公司由史蒂夫·乔布斯在加利福尼亚州创立,它是著名的iPhone制造商。"-graph_kb.feed_text(text) # 提取知识-graph_kb.build_index() # 构建向量索引 (用于后续问答)--# 3. 针对提取出的结构进行问答 (Actionable!)-answer = graph_kb.chat("乔布斯和苹果公司是什么关系?")+# 3. 针对结构化知识直接对话 (具有逻辑支撑的答案)+answer = kb.chat("苹果公司的估值驱动因素有哪些?")print(answer.content)```+---## 🎨 领域模版 (Templates)diff --git a/hyperextract/templates/README.md b/hyperextract/templates/README.mdindex fea8e28..a704e49 100644--- a/hyperextract/templates/README.md+++ b/hyperextract/templates/README.md@@ -63,23 +63,16 @@ The default choice for unstructured text that doesn't fit a specific industry.### 2. `finance` (Finance)Optimized for complex financial relationships, market sentiment, and temporal events.-* **Equity Research Reports**: Detailed analysis by analysts containing ratings, target prices, and valuation logic.+* **Equity Research Reports**: Detailed analysis by analysts containing ratings, target prices, and investment logic.| Template Name | Primitive | Description | Typical Use Case || :--- | :--- | :--- | :--- || **`ResearchNoteSummary`** | `AutoModel` | **Research Core**. Extracts ratings (Buy/Sell), target price, and top-level investment logic. | Report database, screening || **`FinancialForecast`** | `AutoList` | **Financial Estimates**. Extracts projected data (Revenue, EPS, PE) for future years. | Consensus analysis, modeling || **`ValuationLogicMap`** | `AutoGraph` | **Valuation Logic**. Maps causal chains driving stock performance (e.g., New Market -> Growth). | Investment strategy mapping |+| **`FactorInfluenceHypergraph`** | `AutoHypergraph` | **Multi-factor Analysis**. Models complex relationships between macro factors, industry trends, and company metrics. | Quantamental strategy, factor attribution || **`RiskFactorList`** | `AutoList` | **Risk Registry**. Lists specific downside risks (Regulatory, FX, Supply Chain). | Risk monitoring, negatives scan |-* **Earnings Call Transcripts**: Dialogue between management and analysts, rich in performance attribution and strategic outlook.--| Template Name | Primitive | Description | Typical Use Case |-| :--- | :--- | :--- | :--- |-| **`QAExtraction`** | `AutoList` | **Q&A Pairs**. Structures unstructured dialogue into Question-Answer-Speaker sets. | Meeting summaries, sentiment |-| **`GuidanceTracker`** | `AutoModel` | **Outlook Guidance**. Specifically targets quantitative forward-looking guidance (Revenue/Capex). | Expectation management |-| **`PerformanceAttribution`** | `AutoGraph` | **Performance Attribution**. Maps reasons for past performance (e.g., Inflation -> Margin Squeeze). | Attribution analysis |-* **Prospectuses / IPO Filings (S-1)**: Comprehensive disclosures of company history, shareholder structure, and use of proceeds.| Template Name | Primitive | Description | Typical Use Case |diff --git a/hyperextract/templates/README_ZH.md b/hyperextract/templates/README_ZH.mdindex 3c2ed10..594aed3 100644--- a/hyperextract/templates/README_ZH.md+++ b/hyperextract/templates/README_ZH.md@@ -70,16 +70,9 @@| **`ResearchNoteSummary`** | `AutoModel` | **研报核心摘要**。提取评级(买入/卖出)、目标价及核心投资逻辑摘要。 | 研报数据库、快速筛选 || **`FinancialForecast`** | `AutoList` | **财务预测表**。提取分析师对未来几年的营收、EPS、PE 等关键指标预测。 | 一致性预期分析、估值建模 || **`ValuationLogicMap`** | `AutoGraph` | **估值逻辑图**。提取驱动股价变化的因果链条(如:新产品发布 -> 营收增长)。 | 投资逻辑图志、归因分析 |+| **`FactorInfluenceHypergraph`** | `AutoHypergraph` | **多因子影响分析**。建模宏观因素、行业趋势与公司指标之间的复杂关联。 | 量化基本面分析、因子归因 || **`RiskFactorList`** | `AutoList` | **风险清单**。提取报告中的具体下行风险点(监管、汇率、供应链风险等)。 | 风险监控、负面因素扫描 |-* **业绩电话会实录 (Earnings Call Transcripts)**:管理层与分析师的问答记录,包含业绩归因及前瞻性指引。--| 模板名称 | 底层原语 | 描述 | 典型应用场景 |-| :--- | :--- | :--- | :--- |-| **`QAExtraction`** | `AutoList` | **问答对提取**。将非结构化对话整理为结构化的 问题-回答人-回答摘要。 | 纪要整理、多维观点比对 |-| **`GuidanceTracker`** | `AutoModel` | **业绩指引追踪**。专门提取管理层给出的未来业绩承诺或定量展望数据。 | 预期管理、模型更新 |-| **`PerformanceAttribution`** | `AutoGraph` | **业绩归因图**。提取业绩波动背后的原因(如:原料涨价 -> 毛利受到挤压)。 | 经营分析、趋势判断 |-* **招股说明书 (Prospectus/IPO)**:企业上市前的全面披露文件,详述股权结构、募集资金用途等。| 模板名称 | 底层原语 | 描述 | 典型应用场景 |