Skip to content

Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.

更新 .gitignore 文件,添加 tmp/ 目录;重构知识图谱相关类,合并 SimpleGraph 实现至 base.py,删除冗余文件。

yifanfeng9741cf1fcc8ccd
diff --git a/.gitignore b/.gitignore
index f1f9d7d..c8fd3ff 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,5 @@
+tmp/
+
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
diff --git a/README.md b/README.md
index 6e2b753..b8dcb26 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,111 @@
-Hyper-Extract
\ No newline at end of file
+
+# Hyper-Extract 🧠
+
+> **An Intelligent, LLM-Driven Knowledge Container Framework.**
+>
+> 一个基于 LLM 的智能知识提取与演化框架。不仅仅是存储数据,它能“阅读”、“思考”并“进化”。
+
+## 📖 简介 (Introduction)
+
+**Hyper-Extract** 是一个重新定义的非结构化数据处理框架。与传统的 ETL 工具不同,Hyper-Extract 的核心单元是 **Knowledge Container(知识容器)**。
+
+每个容器(如 `SimpleGraph`, `HyperGraph`)都持有一个 **LLM 实例**。它们不仅仅是被动的数据结构,而是具有生命周期的智能体:
+
+1. **Extract (摄入)**: 主动利用 LLM 从文本中提取信息,并执行去重、对齐、合并。
+2. **Evolve (演化)**: 对已有知识进行内省(Introspection)。包括推理新关系、剪枝错误节点、图卷积传播等。
+3. **Dump (产出)**: 将清洗后的高质量知识导出为标准格式(JSON/NetworkX/Neo4j)。
+
+## ✨ 核心特性 (Features)
+
+* **LLM Native**: 容器初始化即绑定 LangChain 模型实例,所有操作皆由大模型驱动。
+* **统一生命周期**: 所有知识类型遵循 `Extract` -> `Evolve` -> `Dump` 的标准范式。
+* **多模态结构支持**:
+ * **Graph**: 标准实体关系图谱 (Nodes + Edges)。
+ * **Hypergraph**: 支持超边(Hyperedge)连接多个节点的复杂语义结构 (HyperRAG 实现)。
+* **类型安全**: 全面基于 Pydantic Schema,确保结构化输出的稳定性。
+
+## 🛠️ 安装 (Installation)
+
+本项目使用 `uv` 进行包管理(也支持标准 pip)。
+
+```bash
+# 克隆项目
+git clone https://github.com/your-username/hyper-extract.git
+cd hyper-extract
+
+# 使用 uv 安装依赖 (推荐)
+uv sync
+
+# 或者使用 pip
+pip install -r requirements.txt
+```
+
+## 🚀 快速开始 (Quick Start)
+
+### 1. 定义与初始化
+
+```python
+import asyncio
+from langchain_openai import ChatOpenAI
+from src.knowledge.graph.simple_graph import SimpleGraph
+
+async def main():
+ # 1. 准备大模型实例
+ llm = ChatOpenAI(model="gpt-4o", temperature=0)
+
+ # 2. 初始化智能容器
+ # 容器就像一个空的大脑,准备好接收信息
+ kg = SimpleGraph(llm=llm)
+
+ # 3. Extract: 摄入知识
+ text = "Elon Musk co-founded OpenAI in 2015. He later left the board in 2018."
+ print(f"正在提取: {text}...")
+ await kg.extract(text)
+
+ # 4. Evolve: 知识演化 (可选)
+ # 例如:推断隐含关系或清理孤立节点
+ await kg.evolve()
+
+ # 5. Dump: 导出结果
+ data = kg.dump(format="json")
+ print("最终知识图谱:", data)
+
+if __name__ == "__main__":
+ asyncio.run(main())
+```
+
+## 📂 项目结构 (Project Structure)
+
+```text
+hyper-extract/
+├── src/
+│ ├── core/ # 核心抽象定义
+│ ├── driver/ # LLM 驱动与适配器
+│ ├── schema/ # Pydantic 数据结构定义 (GraphSchema, etc.)
+│ └── knowledge/ # 知识容器实现
+│ ├── base.py # [核心] BaseKnowledge 基类
+│ ├── graph/ # 普通图谱实现
+│ │ ├── simple_graph.py # 基础图提取
+│ │ └── kg_gen_graph.py # 复杂生成式图谱
+│ └── hypergraph/ # 超图实现
+│ ├── simple_hypergraph.py
+│ └── hyperrag_hypergraph.py
+├── tests/ # 测试用例
+├── main.py # 入口示例
+├── pyproject.toml # 项目配置
+└── uv.lock # 依赖锁定
+```
+
+## 🧩 扩展指南 (Development)
+
+如果你想添加一种新的知识结构(例如“思维导图”):
+
+1. 继承 `src.knowledge.base.BaseKnowledge`。
+2. 定义你的数据 Schema (Pydantic)。
+3. 实现 **`extract`**: 定义如何用 Prompt 将文本转化为你的 Schema。
+4. 实现 **`evolve`** (可选): 定义如何优化你的数据结构。
+5. 实现 **`dump`**: 定义导出格式。
+
+## 📄 License
+
+MIT License
\ No newline at end of file
diff --git a/src/knowledge/graph/base.py b/src/knowledge/graph/base.py
index e69de29..5999216 100644
--- a/src/knowledge/graph/base.py
+++ b/src/knowledge/graph/base.py
@@ -0,0 +1,80 @@
+from ..base import BaseKnowledge
+from typing import List, Optional, Dict, Any
+from pydantic import BaseModel, Field
+
+
+class Entity(BaseModel):
+ """Represents a node in the knowledge graph."""
+
+ id: str = Field(
+ ...,
+ description="The unique name or identifier of the entity (e.g., 'Elon Musk').",
+ )
+ type: str = Field(
+ "Unknown",
+ description="The category of the entity (e.g., 'Person', 'Organization', 'Concept').",
+ )
+ description: Optional[str] = Field(
+ None, description="A brief summary of the entity found in the text."
+ )
+ # Using a dict allows storing extra attributes without changing the schema structure
+ properties: Dict[str, Any] = Field(
+ default_factory=dict,
+ description="Additional attributes (e.g., {'age': 50, 'status': 'active'}).",
+ )
+
+
+class Relationship(BaseModel):
+ """Represents an edge between two entities."""
+
+ source: str = Field(..., description="The id of the source entity.")
+ target: str = Field(..., description="The id of the target entity.")
+ type: str = Field(
+ ..., description="The type of the relationship (e.g., 'FOUNDED', 'LOCATED_IN')."
+ )
+ description: Optional[str] = Field(
+ None, description="Contextual details about this relationship."
+ )
+ weight: float = Field(
+ 1.0,
+ description="The confidence score or strength of the connection (0.0 to 1.0).",
+ )
+
+
+class GraphSchema(BaseModel):
+ """The container for the entire graph structure."""
+
+ nodes: List[Entity] = Field(default_factory=list)
+ edges: List[Relationship] = Field(default_factory=list)
+
+ def get_node_ids(self) -> set[str]:
+ """Helper to get all current node IDs for quick lookup."""
+ return {n.id for n in self.nodes}
+
+
+class SimpleGraph(BaseKnowledge[GraphSchema]):
+ def _get_schema_class(self):
+ return GraphSchema
+
+ async def extract(self, text: str, **kwargs):
+ # 1. 构造 prompt
+ prompt = f"Extract nodes and edges from: {text}"
+
+ # 2. 调用自带的 LLM
+ response = await self.llm.ainvoke(prompt)
+
+ # 3. 解析结果 (假设 parsed_data 是解析后的 GraphSchema 对象)
+ parsed_data = self._parse(response)
+
+ # 4. 【直接在这里做合并逻辑】
+ # 简单的去重合并
+ existing_node_ids = {n.id for n in self.data.nodes}
+ for node in parsed_data.nodes:
+ if node.id not in existing_node_ids:
+ self.data.nodes.append(node)
+ self.data.edges.extend(parsed_data.edges)
+
+ def dump(self, format="json"):
+ if format == "json":
+ return self.data.model_dump_json()
+ return self.data.model_dump()
diff --git a/src/knowledge/graph/simple_graph.py b/src/knowledge/graph/simple_graph.py
deleted file mode 100644
index 5999216..0000000
--- a/src/knowledge/graph/simple_graph.py
+++ /dev/null
@@ -1,80 +0,0 @@
-from ..base import BaseKnowledge
-from typing import List, Optional, Dict, Any
-from pydantic import BaseModel, Field
-
-
-class Entity(BaseModel):
- """Represents a node in the knowledge graph."""
-
- id: str = Field(
- ...,
- description="The unique name or identifier of the entity (e.g., 'Elon Musk').",
- )
- type: str = Field(
- "Unknown",
- description="The category of the entity (e.g., 'Person', 'Organization', 'Concept').",
- )
- description: Optional[str] = Field(
- None, description="A brief summary of the entity found in the text."
- )
- # Using a dict allows storing extra attributes without changing the schema structure
- properties: Dict[str, Any] = Field(
- default_factory=dict,
- description="Additional attributes (e.g., {'age': 50, 'status': 'active'}).",
- )
-
-
-class Relationship(BaseModel):
- """Represents an edge between two entities."""
-
- source: str = Field(..., description="The id of the source entity.")
- target: str = Field(..., description="The id of the target entity.")
- type: str = Field(
- ..., description="The type of the relationship (e.g., 'FOUNDED', 'LOCATED_IN')."
- )
- description: Optional[str] = Field(
- None, description="Contextual details about this relationship."
- )
- weight: float = Field(
- 1.0,
- description="The confidence score or strength of the connection (0.0 to 1.0).",
- )
-
-
-class GraphSchema(BaseModel):
- """The container for the entire graph structure."""
-
- nodes: List[Entity] = Field(default_factory=list)
- edges: List[Relationship] = Field(default_factory=list)
-
- def get_node_ids(self) -> set[str]:
- """Helper to get all current node IDs for quick lookup."""
- return {n.id for n in self.nodes}
-
-
-class SimpleGraph(BaseKnowledge[GraphSchema]):
- def _get_schema_class(self):
- return GraphSchema
-
- async def extract(self, text: str, **kwargs):
- # 1. 构造 prompt
- prompt = f"Extract nodes and edges from: {text}"
-
- # 2. 调用自带的 LLM
- response = await self.llm.ainvoke(prompt)
-
- # 3. 解析结果 (假设 parsed_data 是解析后的 GraphSchema 对象)
- parsed_data = self._parse(response)
-
- # 4. 【直接在这里做合并逻辑】
- # 简单的去重合并
- existing_node_ids = {n.id for n in self.data.nodes}
- for node in parsed_data.nodes:
- if node.id not in existing_node_ids:
- self.data.nodes.append(node)
- self.data.edges.extend(parsed_data.edges)
-
- def dump(self, format="json"):
- if format == "json":
- return self.data.model_dump_json()
- return self.data.model_dump()
diff --git a/src/knowledge/hypergraph/simple_hypergraph.py b/src/knowledge/hypergraph/simple_hypergraph.py
deleted file mode 100644
index e69de29..0000000