Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: 将 show_progress 参数重命名为 verbose,增强日志记录功能并更新相关文档
yifanfeng97d75643319362
diff --git a/README.md b/README.mdindex 7d78bd7..6fea8e1 100644--- a/README.md+++ b/README.md@@ -103,7 +103,7 @@ a_article = AutoModel(embedder=embedder,chunk_size=2000, # Auto-split long textsmax_workers=5, # Concurrent processing- show_progress=True+ verbose=True)```diff --git a/examples/auto_graph_demo.py b/examples/auto_graph_demo.pyindex 397cc2a..7e72fcb 100644--- a/examples/auto_graph_demo.py+++ b/examples/auto_graph_demo.py@@ -303,7 +303,7 @@ def run_demo():"重要规则:互动中的 source 和 target 必须是你刚才提取过的实体名称。""例如,如果之前提取了节点'深渊魔龙',那么攻击BOSS的互动 target 必须填'深渊魔龙'。"),- show_progress=True,+ verbose=True,)print(f"\n📖 正在阅读游戏论坛爆料帖...")diff --git a/examples/auto_hypergraph_demo.py b/examples/auto_hypergraph_demo.pyindex 97638d9..b7c499c 100644--- a/examples/auto_hypergraph_demo.py+++ b/examples/auto_hypergraph_demo.py@@ -154,7 +154,7 @@ def main():llm_client=llm,embedder=embedder,extraction_mode="two_stage", # 推荐:先识别实体,再梳理故事线- show_progress=True,+ verbose=True,# 使用中文 Prompt 引导 LLMprompt_for_node_extraction=("你的任务是从故事文本中提取所有重要的实体节点。\n"diff --git a/examples/auto_list_demo.py b/examples/auto_list_demo.pyindex 9f077ac..744be29 100644--- a/examples/auto_list_demo.py+++ b/examples/auto_list_demo.py@@ -113,7 +113,7 @@ def main():chunk_size=500, # 设置小的 chunk_size 来测试多 chunk 提取chunk_overlap=50,max_workers=5,- show_progress=True,+ verbose=True,)# 4. 提取知识 (使用 feed_text 模式)diff --git a/examples/auto_model_demo.py b/examples/auto_model_demo.pyindex 3d84885..0a47a7c 100644--- a/examples/auto_model_demo.py+++ b/examples/auto_model_demo.py@@ -162,7 +162,7 @@ def main():chunk_size=500, # 设置小的 chunk_size 来测试多 chunk 提取chunk_overlap=50,max_workers=5,- show_progress=True,+ verbose=True,)# 4. 提取知识 (使用 feed_text 模式)diff --git a/examples/hyper_rag_demo.py b/examples/hyper_rag_demo.pynew file mode 100644index 0000000..2d0c8d8--- /dev/null+++ b/examples/hyper_rag_demo.py@@ -0,0 +1,257 @@+"""+Hyper-RAG Demo: Multi-entity Relationship Extraction with Hypergraph Support++This demo demonstrates:+1. Extracting entities and multi-entity relationships from a complex sci-fi story.+2. Analysis of complex n-ary relationships (Hyperedges).+3. Semantic Search over the constructed Hypergraph (Q&A).+"""++import os+import sys++# 添加项目根目录到路径+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddings++from hyperextract.hypergraphs import Hyper_RAG++load_dotenv()++# ============================================================================+# Sample Story for Extraction (Shared with AutoHypergraph Demo)+# ============================================================================++STORY = """+【银河纪元 3042年 - 赤红星云事件报告】++危机在这一周达到了顶峰。整个银河系似乎都陷入了暗流涌动的政治漩涡。++【第一阶段:秘密接触】+臭名昭著的星际海盗黑蛇驾驶着他的旗舰复仇号,在泰坦空间站秘密会见了银河联邦的背叛者议员 Valerius。+目击者称,两人在午夜时分登上了空间站的顶层禁区。据情报部门推测,这是一次肮脏的交易——+双方达成了针对皇室的同盟关系。参与此会议的还有 Valerius 手下的特务头目 Shadow,+他负责处理所有不可告人的细节。这个同盟的形成标志着大麻烦即将到来。++【第二阶段:叛变与夺权】+在这次会议的两天后,事态急剧恶化。黑蛇这种疯狂的家伙竟然在极光星区伏击了皇家护卫队。+这场突然的冲突中,皇家护卫队的队长 Leona 与副队长 Marcus 奋力抵抗,+但他们陷入了黑蛇手下的精锐战士的包围。这场激战导致队长 Leona 身受重伤。+而黑蛇的目标不是为了杀戮,而是为了夺取一件神圣的物品——传说中的虚空水晶。+这枚水晶散发着幽蓝的光芒,据说能够打开先祖之门的第一道密封。+黑蛇在战斗中成功夺取了水晶,但 Leona 在临终前用最后的力量将其重新夺回。++【第三阶段:流亡与隐藏】+Leona 被迫带着虚空水晶,从极光星区逃往荒凉的废土行星 Z-9。+在这个被遗忘的角落,她向仅有的几个信任者求救:还有副队长 Marcus 以及隐秘组织"地下抵抗军"的领导人 Cipher。+同时,议员 Valerius 非法持有了开启先祖之门的钥匙——这把钥匙本应被安全保管在皇家金库中。+他现在藏匿在浮空都市中,与黑蛇遥相呼应,计划着下一步的政变。++【第四阶段:势力重组】+银河联邦的最高委员会得知了这一系列事件,决定采取行动。+高级顾问 Artemis 被任命为特使,她与 Leona 在废土行星 Z-9 进行了秘密会晤,+讨论如何制止这场即将到来的危机。同时,地下抵抗军的 Cipher 也加入了这次会议,+三人共同制定了一个大胆的计划。++【第五阶段:最后的筹码】+虽然黑蛇未能获得虚空水晶,但他已经从 Valerius 那里获得了先祖之门钥匙的复制品。+根据情报,这个复制品目前被秘密运往到黑蛇的秘密基地——位于辐射荒漠中的要塞。+参与此次运送的还有 Shadow 和黑蛇手下的精锐战士群体。+与此同时,Leona 和 Marcus 正在准备一场反击,他们计划在虚空水晶的力量下重新夺回局势的控制权。++目前局势:+- 黑蛇:手持复制品钥匙,掌控旗舰复仇号,计划进行最终的攻击+- 议员 Valerius:躲在浮空都市中,与黑蛇保持联系+- Leona:守护虚空水晶,在废土行星 Z-9 养伤,准备反击+- 银河联邦:通过特使 Artemis 支持 Leona,正在调集力量+- 地下抵抗军:Cipher 领导,与 Leona 和 Marcus 联手,准备对抗黑蛇和 Valerius 的同盟++这场战争的走向仍不确定。虚空水晶、先祖之门、以及隐藏在这一切背后的秘密,+都将在接下来的日子里浮出水面。银河的命运,可能就在这些关键人物和物品的碰撞中决定。+"""+++if __name__ == "__main__":+ # ============================================================================+ # Initialize the Hyper-RAG system with LLM and Embeddings+ # ============================================================================+ print("=" * 80)+ print("Initializing Hyper-RAG System...")+ print("=" * 80)++ # llm_client = ChatOpenAI(model="gpt-4o-mini", temperature=0)+ llm_client = ChatOpenAI(+ model="gpt-4o-mini",+ extra_body={+ "system": "You are a helpful assistant. Always respond in Chinese."+ },+ temperature=0,+ )+ embedder = OpenAIEmbeddings(model="text-embedding-3-small")++ rag = Hyper_RAG(+ llm_client=llm_client,+ embedder=embedder,+ verbose=True,+ chunk_size=1000,+ )++ # ============================================================================+ # Feed text to the system+ # ============================================================================+ print("\nFeeding story to the system...")+ rag.feed_text(STORY)+ print("✓ Story processed successfully\n")++ # ============================================================================+ # Display Extracted Entities+ # ============================================================================+ print("=" * 80)+ print("EXTRACTED ENTITIES")+ print("=" * 80)++ for node in rag.nodes:+ print(f"\n【{node.name}】(Type: {node.type})")+ print(f" Description: {node.description}")++ print(f"\n✓ Total entities extracted: {len(rag.nodes)}\n")++ # ============================================================================+ # Display Extracted Hyperedges+ # ============================================================================+ print("=" * 80)+ print("EXTRACTED HYPEREDGES")+ print("=" * 80)++ # 分离低阶边(2个参与者)和高阶边(3个以上参与者)+ low_order_edges = [e for e in rag.edges if len(e.participants) == 2]+ high_order_edges = [e for e in rag.edges if len(e.participants) > 2]++ print(f"\n--- Low-Order Edges (Pairwise) ---")+ for i, edge in enumerate(low_order_edges, 1):+ print(f"\n{i}. Participants: {' <-> '.join(edge.participants)}")+ print(f" Description: {edge.description}")+ keywords_str = ", ".join(edge.keywords) if edge.keywords else "N/A"+ print(f" Keywords: {keywords_str}")+ print(f" Strength: {edge.strength}/10")++ print(f"\n--- High-Order Edges (Hyperedges) ---")+ for i, edge in enumerate(high_order_edges, 1):+ print(f"\n{i}. Participants: {' <-> '.join(edge.participants)}")+ print(f" Description: {edge.description}")+ keywords_str = ", ".join(edge.keywords) if edge.keywords else "N/A"+ print(f" Keywords: {keywords_str}")+ print(f" Strength: {edge.strength}/10")++ print(+ f"\n✓ Total edges extracted: {len(rag.edges)} ({len(low_order_edges)} low-order, {len(high_order_edges)} high-order)\n"+ )++ # ============================================================================+ # Statistics+ # ============================================================================+ print("=" * 80)+ print("STATISTICS")+ print("=" * 80)++ # Entity type distribution+ type_counts = {}+ for node in rag.nodes:+ type_counts[node.type] = type_counts.get(node.type, 0) + 1++ print("\nEntity Type Distribution:")+ for entity_type, count in sorted(type_counts.items()):+ print(f" {entity_type}: {count}")++ # Average edge strength+ if rag.edges:+ avg_strength = sum(e.strength for e in rag.edges) / len(rag.edges)+ print(f"\nAverage Edge Strength: {avg_strength:.2f}/10")++ # Centrality (number of relationships per entity)+ print("\nEntity Centrality (number of relationships):")+ centrality = {}+ for edge in rag.edges:+ for participant in edge.participants:+ centrality[participant] = centrality.get(participant, 0) + 1++ for entity, count in sorted(centrality.items(), key=lambda x: x[1], reverse=True)[+ :10+ ]:+ print(f" {entity}: {count}")++ # ============================================================================+ # Build Semantic Index for Q&A+ # ============================================================================+ print("\n" + "=" * 80)+ print("Building Semantic Index...")+ print("=" * 80)++ rag.build_index()+ print("✓ Semantic index built successfully\n")++ # ============================================================================+ # Q&A: Semantic Search over Hypergraph+ # ============================================================================+ print("=" * 80)+ print("Q&A: SEMANTIC SEARCH")+ print("=" * 80)++ queries = [+ "虚空水晶现在在哪里?",+ "黑蛇和议员Valerius有什么阴谋?",+ "谁参与了泰坦空间站的秘密会议?",+ "Leona目前的状况如何?",+ ]++ for i, query in enumerate(queries, 1):+ print(f"\n【问题 {i}】{query}")+ print("-" * 60)++ try:+ results = rag.search_edges(query, top_k=3)++ if results:+ for j, result in enumerate(results, 1):+ edge = result[0] if isinstance(result, tuple) else result+ score = result[1] if isinstance(result, tuple) else 1.0++ print(f"\n {j}. [Score: {score:.3f}]")+ print(f" Participants: {' <-> '.join(edge.participants)}")+ print(f" Description: {edge.description}")+ print(f" Strength: {edge.strength}/10")+ else:+ print(" (No relevant relationships found)")+ except Exception as e:+ print(f" Search error: {str(e)}")++ # ============================================================================+ # Save Results+ # ============================================================================+ print("\n" + "=" * 80)+ print("Saving Results...")+ print("=" * 80)++ try:+ output_dir = "temp/hyper_rag_demo"+ rag.dump(output_dir)+ print(f"✓ Results saved to {output_dir}/")++ # List saved files+ saved_files = []+ if os.path.exists(output_dir):+ for root, dirs, files in os.walk(output_dir):+ for file in files:+ rel_path = os.path.relpath(os.path.join(root, file), output_dir)+ saved_files.append(rel_path)++ if saved_files:+ print("\n Saved files:")+ for f in saved_files:+ print(f" - {f}")+ except Exception as e:+ print(f"⚠ Warning: Could not save results: {str(e)}")++ print("\n" + "=" * 80 + "\n")diff --git a/examples/kg_gen_demo.py b/examples/kg_gen_demo.pyindex b7ffbf8..e1b9a7b 100644--- a/examples/kg_gen_demo.py+++ b/examples/kg_gen_demo.py@@ -73,7 +73,7 @@ def main():kg = KG_Gen(llm_client=llm,embedder=embedder,- show_progress=True,+ verbose=True,)# 2. 提取@@ -171,6 +171,34 @@ def main():for entity, count in entity_degree.most_common(5):print(f" • {entity}: 参与 {count} 个关系")+ # ============================================================================+ # Save Results+ # ============================================================================+ print("\n" + "=" * 70)+ print("Saving Results...")+ print("=" * 70)++ try:+ import os+ output_dir = "temp/kg_gen_demo"+ kg.dump(output_dir)+ print(f"✓ Results saved to {output_dir}/")++ # List saved files+ saved_files = []+ if os.path.exists(output_dir):+ for root, dirs, files in os.walk(output_dir):+ for file in files:+ rel_path = os.path.relpath(os.path.join(root, file), output_dir)+ saved_files.append(rel_path)++ if saved_files:+ print("\n Saved files:")+ for f in saved_files:+ print(f" - {f}")+ except Exception as e:+ print(f"⚠ Warning: Could not save results: {str(e)}")+print("\n" + "=" * 70)print("✨ 演示完成!")print("=" * 70)diff --git a/hyperextract/core/base.py b/hyperextract/core/base.pyindex acc2f88..4102d68 100644--- a/hyperextract/core/base.py+++ b/hyperextract/core/base.py@@ -42,7 +42,7 @@ class BaseAutoType(ABC, Generic[T]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,):"""Initialize the knowledge object with schema and processing configuration.@@ -54,7 +54,7 @@ class BaseAutoType(ABC, Generic[T]):chunk_size: Maximum chunk size for splitting long texts.chunk_overlap: Number of overlapping characters between chunks.max_workers: Maximum number of concurrent extraction tasks.- show_progress: Whether to display progress information during processing.+ verbose: Whether to display detailed execution logs and progress information."""self._data_schema = data_schemaself.llm_client = llm_client@@ -63,7 +63,7 @@ class BaseAutoType(ABC, Generic[T]):self.chunk_size = chunk_sizeself.chunk_overlap = chunk_overlapself.max_workers = max_workers- self.show_progress = show_progress+ self.verbose = verbose# Initialize text splitter for chunking long documentsself.text_splitter = RecursiveCharacterTextSplitter(@@ -97,7 +97,7 @@ class BaseAutoType(ABC, Generic[T]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,)@abstractmethoddiff --git a/hyperextract/core/list.py b/hyperextract/core/list.pyindex feea855..84f9107 100644--- a/hyperextract/core/list.py+++ b/hyperextract/core/list.py@@ -62,7 +62,7 @@ class AutoList(BaseAutoType[AutoListSchema[Item]], Generic[Item]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,fields_for_index: List[str] | None = None,):"""Initialize AutoList with item schema and configuration.@@ -75,7 +75,7 @@ class AutoList(BaseAutoType[AutoListSchema[Item]], Generic[Item]):chunk_size: Maximum characters per chunk for long texts.chunk_overlap: Overlapping characters between adjacent chunks.max_workers: Maximum concurrent extraction tasks.- show_progress: Whether to log progress information.+ verbose: Whether to log progress information.fields_for_index: Optional list of field names to include in vector index.If None, all fields are indexed."""@@ -107,7 +107,7 @@ class AutoList(BaseAutoType[AutoListSchema[Item]], Generic[Item]):chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,)def _default_prompt(self) -> str:@@ -141,7 +141,7 @@ class AutoList(BaseAutoType[AutoListSchema[Item]], Generic[Item]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,fields_for_index=self.fields_for_index,)diff --git a/hyperextract/core/model.py b/hyperextract/core/model.pyindex da43860..49963e7 100644--- a/hyperextract/core/model.py+++ b/hyperextract/core/model.py@@ -43,7 +43,7 @@ class AutoModel(BaseAutoType[T]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,**kwargs,):"""Initialize AutoModel with schema and configuration.@@ -60,7 +60,7 @@ class AutoModel(BaseAutoType[T]):chunk_size: Maximum characters per chunk for long texts.chunk_overlap: Overlapping characters between adjacent chunks.max_workers: Maximum concurrent extraction tasks.- show_progress: Whether to log progress information.+ verbose: Whether to log progress information."""# Store strategy before calling super().__init__self._strategy_or_merger = strategy_or_merger@@ -73,7 +73,7 @@ class AutoModel(BaseAutoType[T]):chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,**kwargs,)@@ -110,7 +110,7 @@ class AutoModel(BaseAutoType[T]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,)def _default_prompt(self) -> str:@@ -343,7 +343,7 @@ class AutoModel(BaseAutoType[T]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,)# Set items from both units@@ -375,7 +375,7 @@ class AutoModel(BaseAutoType[T]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,)# Prepend self to other's itemsdiff --git a/hyperextract/core/set.py b/hyperextract/core/set.pyindex da80ca2..2335aa1 100644--- a/hyperextract/core/set.py+++ b/hyperextract/core/set.py@@ -95,7 +95,7 @@ class AutoSet(BaseAutoType[AutoSetSchema[Item]], Generic[Item]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,fields_for_index: List[str] | None = None,**kwargs: Any,):@@ -113,7 +113,7 @@ class AutoSet(BaseAutoType[AutoSetSchema[Item]], Generic[Item]):chunk_size: Maximum characters per chunk for long texts.chunk_overlap: Overlapping characters between adjacent chunks.max_workers: Maximum concurrent extraction tasks.- show_progress: Whether to log progress information.+ verbose: Whether to display detailed execution logs and progress information.fields_for_index: Optional list of field names in item_schema to include in vector index.If None, all text fields are indexed by default.Useful for optimizing search on complex schemas.@@ -166,7 +166,7 @@ class AutoSet(BaseAutoType[AutoSetSchema[Item]], Generic[Item]):llm_client=llm_client,embedder=embedder,strategy_or_merger=self._merger,- verbose=show_progress,+ verbose=verbose,fields_for_index=fields_for_index, # Pass field selection to OMem)@@ -178,7 +178,7 @@ class AutoSet(BaseAutoType[AutoSetSchema[Item]], Generic[Item]):chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,)# ==================== Override Instance Creation ====================@@ -201,7 +201,7 @@ class AutoSet(BaseAutoType[AutoSetSchema[Item]], Generic[Item]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,fields_for_index=self.fields_for_index, # Persist index field configuration)diff --git a/hyperextract/graphs/base.py b/hyperextract/graphs/base.pyindex 8273f6f..8cd9f83 100644--- a/hyperextract/graphs/base.py+++ b/hyperextract/graphs/base.py@@ -121,7 +121,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,node_fields_for_index: List[str] | None = None,edge_fields_for_index: List[str] | None = None,**kwargs: Any,@@ -145,7 +145,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):chunk_size: Maximum characters per chunk.chunk_overlap: Overlapping characters between chunks.max_workers: Maximum concurrent extraction tasks.- show_progress: Whether to log progress.+ verbose: Whether to log progress.node_fields_for_index: Optional list of field names in node_schema to include in vector index.If None, all text fields are indexed by default.Example: ['name', 'description'] (only index these node fields)@@ -224,7 +224,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):llm_client=llm_client,embedder=embedder,strategy_or_merger=self.node_merger,- verbose=show_progress,+ verbose=verbose,fields_for_index=node_fields_for_index, # Pass node field selection to OMem)@@ -234,7 +234,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):llm_client=llm_client,embedder=embedder,strategy_or_merger=self.edge_merger,- verbose=show_progress,+ verbose=verbose,fields_for_index=edge_fields_for_index, # Pass edge field selection to OMem)@@ -247,7 +247,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,)def _default_prompt(self) -> str:@@ -279,7 +279,7 @@ class AutoGraph(BaseAutoType[AutoGraphSchema[Node, Edge]], Generic[Node, Edge]):chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,node_fields_for_index=self.node_fields_for_index, # Persist node index field configurationedge_fields_for_index=self.edge_fields_for_index, # Persist edge index field configuration)diff --git a/hyperextract/graphs/kg_gen.py b/hyperextract/graphs/kg_gen.pyindex 44118c9..c456c6c 100644--- a/hyperextract/graphs/kg_gen.py+++ b/hyperextract/graphs/kg_gen.py@@ -7,11 +7,8 @@ Prompts are adapted from kg_gen/steps to ensure consistency with the originalknowledge graph extraction logic."""-import unicodedata-from typing import List, Tuple, Dict-from pydantic import BaseModel, Field-from semhash import SemHash+from pydantic import BaseModel, Fieldfrom sentence_transformers import SentenceTransformerfrom langchain_core.language_models import BaseChatModelfrom langchain_core.embeddings import Embeddings@@ -113,7 +110,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,):"""Initialize KGGenGraph.@@ -123,7 +120,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):chunk_size: Characters per chunkchunk_overlap: Overlapping characters between chunksmax_workers: Max concurrent extraction workers- show_progress: Whether to show progress logging+ verbose: Display detailed execution logs and progress information"""# 1. Define Key Extractors (critical for deduplication)@@ -161,7 +158,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,)def _deduplicate_graph(diff --git a/hyperextract/hypergraphs/__init__.py b/hyperextract/hypergraphs/__init__.pyindex 4a575e4..29ee874 100644--- a/hyperextract/hypergraphs/__init__.py+++ b/hyperextract/hypergraphs/__init__.py@@ -1,5 +1,7 @@from .base import AutoHypergraph+from .hyper_rag import Hyper_RAG__all__ = ["AutoHypergraph",+ "Hyper_RAG",]diff --git a/hyperextract/hypergraphs/base.py b/hyperextract/hypergraphs/base.pyindex eb74af7..eb4ea99 100644--- a/hyperextract/hypergraphs/base.py+++ b/hyperextract/hypergraphs/base.py@@ -117,7 +117,7 @@ class AutoHypergraph(chunk_size: int = 2000,chunk_overlap: int = 200,max_workers: int = 10,- show_progress: bool = True,+ verbose: bool = True,node_fields_for_index: List[str] | None = None,edge_fields_for_index: List[str] | None = None,**kwargs: Any,@@ -148,7 +148,7 @@ class AutoHypergraph(chunk_size: Maximum characters per chunk.chunk_overlap: Overlapping characters between chunks.max_workers: Maximum concurrent extraction tasks.- show_progress: Whether to log progress.+ verbose: Whether to display detailed execution logs and progress information.node_fields_for_index: Optional list of field names in node_schema to include in vector index.If None, all text fields are indexed by default.Example: ['name', 'properties'] (only index these node fields)@@ -167,12 +167,12 @@ class AutoHypergraph(self.chunk_size = chunk_sizeself.chunk_overlap = chunk_overlapself.max_workers = max_workers- self.show_progress = show_progress+ self.verbose = verboseself.node_fields_for_index = node_fields_for_indexself.edge_fields_for_index = edge_fields_for_indexself.max_workers = max_workers- self.show_progress = show_progress+ self.verbose = verbose# Initialize promptsself.node_prompt = prompt_for_node_extraction or self._default_node_prompt()@@ -227,7 +227,7 @@ class AutoHypergraph(llm_client=llm_client,embedder=embedder,strategy_or_merger=self.node_merger,- verbose=show_progress,+ verbose=verbose,fields_for_index=node_fields_for_index, # Pass node field selection to OMem)@@ -237,7 +237,7 @@ class AutoHypergraph(llm_client=llm_client,embedder=embedder,strategy_or_merger=self.edge_merger,- verbose=show_progress,+ verbose=verbose,fields_for_index=edge_fields_for_index, # Pass edge field selection to OMem)@@ -250,7 +250,7 @@ class AutoHypergraph(chunk_size=chunk_size,chunk_overlap=chunk_overlap,max_workers=max_workers,- show_progress=show_progress,+ verbose=verbose,)# ==================== Prompts ====================@@ -260,9 +260,8 @@ class AutoHypergraph(This is the primary prompt used when extraction_mode is 'one_stage'."""- return self._default_hypergraph_prompt+ return self._default_hypergraph_prompt()- @propertydef _default_hypergraph_prompt(self) -> str:"""Default prompt for one-stage hypergraph extraction (nodes + edges together).@@ -326,7 +325,7 @@ class AutoHypergraph(chunk_size=self.chunk_size,chunk_overlap=self.chunk_overlap,max_workers=self.max_workers,- show_progress=self.show_progress,+ verbose=self.verbose,node_fields_for_index=self.node_fields_for_index, # Persist node index field configurationedge_fields_for_index=self.edge_fields_for_index, # Persist edge index field configuration)@@ -418,7 +417,7 @@ class AutoHypergraph(else:chunks = self.text_splitter.split_text(text)- if self.show_progress:+ if self.verbose:logger.info(f"Extracting from {len(chunks)} chunks...")# 2. Batch Extract Nodesdiff --git a/hyperextract/hypergraphs/hyper_rag.py b/hyperextract/hypergraphs/hyper_rag.pynew file mode 100644index 0000000..81a59c8--- /dev/null+++ b/hyperextract/hypergraphs/hyper_rag.py@@ -0,0 +1,202 @@+"""+Hyper-RAG: Hypergraph-based Retrieval-Augmented Generation+Extracts and manages multi-entity relationships with support for n-ary hyperedges.+"""++from typing import List+from pydantic import BaseModel, Field+from hyperextract.hypergraphs.base import AutoHypergraph+from langchain_core.language_models import BaseChatModel+from langchain_core.embeddings import Embeddings+from ontomem.merger import CustomRuleMerger++# ============================================================================+# Node Schema+# ============================================================================+++class NodeSchema(BaseModel):+ """Represents an entity extracted from the source text."""++ name: str = Field(..., description="Name of the entity, capitalized if in English")+ type: str = Field(+ ...,+ description="Entity type (person, organization, geo, event, role, concept, etc.)",+ )+ description: str = Field(+ ...,+ description="Comprehensive description of the entity's attributes and activities",+ )+++# ============================================================================+# Edge Schema+# ============================================================================+++class EdgeSchema(BaseModel):+ """Represents a relationship or event connecting multiple entities (low-order or high-order)."""++ participants: List[str] = Field(+ ..., description="Names of entities involved in this relationship"+ )+ description: str = Field(+ ..., description="Detailed explanation of the relationship or event"+ )+ keywords: List[str] = Field(+ ...,+ description="List of keywords summarizing the relationship themes (e.g., ['conflict', 'trade', 'alliance'])",+ )+ strength: int = Field(+ ...,+ ge=1,+ le=10,+ description="Numerical score indicating relationship strength (1-10)",+ )+++# ============================================================================+# Extraction Prompts+# ============================================================================++HYPER_RAG_NODE_EXTRACTION_PROMPT = """+-Goal-+Identify relevant entities from the text.+Entities will serve as participants in complex events later.\n\n+### Source Text:+"""++HYPER_RAG_EDGE_EXTRACTION_PROMPT = """+-Goal-+You are an expert hypergraph knowledge extraction assistant.+Extract "Hyperedges" that represent relationships, events, or thematic groupings involving MULTIPLE entities (2 or more) simultaneously.++-Definition-+A Hyperedge is a unified concept for any connection.+It treats all involved entities as **participants** in a shared context+(e.g., a conversation, a joint mission, a conflict, or a shared concept).++-Constraints-+1. You MUST ONLY use names from the "Allowed Entities" list provided below.+2. Do NOT create edges for entities that represent strictly different concepts with no interaction.+3. Ensure every edge has at least 2 participants.++"""++# ============================================================================+# Merge Templates for LLM.CUSTOM_RULE Strategy+# ============================================================================++NODE_MERGE_TEMPLATE = """You are an intelligent data merging assistant.+You will receive a list of objects representing the same Entity.++Your task is to merge them into a SINGLE object exactly matching the schema.++Merge strategy:+1. **name/type**: Keep the most frequent or precise value.+2. **description**: Synthesize a single, comprehensive description containing all unique details from the input descriptions. Write it in the third person. Resolve any contradictions coherently.+"""++EDGE_MERGE_TEMPLATE = """You are an intelligent data merging assistant.+You will receive a list of objects representing the same Hyperedge (Relationship/Event).++Your task is to merge them into a SINGLE object exactly matching the schema.++Merge strategy:+1. **participants**: Keep the list (they should be identical for the same hyperedge key).+2. **description**: Synthesize a single, comprehensive description covering the relationship dynamics from all inputs. Write it in the third person.+3. **keywords**: Combine keyword lists from all inputs, removing duplicates.+4. **strength**: Calculate the average of the input strengths (round to nearest integer).+"""++# ============================================================================+# Hyper_RAG Class+# ============================================================================+++class Hyper_RAG(AutoHypergraph[NodeSchema, EdgeSchema]):+ """+ Hyper-RAG: Hypergraph-based Retrieval-Augmented Generation++ Extracts multi-entity relationships (hyperedges) from text documents.++ Features:+ - Two-stage extraction: Entities first, then low-order (binary) and high-order (n-ary) relationships.+ - Custom Key Extractors: Precise deduplication using name-based node keys and sorted participant tuples for edges.+ - Hyperedge Support: Handles complex n-ary relationships connecting multiple entities simultaneously.+ - Structured Knowledge Representation: Pydantic-based Node and Edge schemas with comprehensive attributes.+ - Advanced Indexing: Optimized field-level indexing for efficient semantic search and retrieval."""++ def __init__(+ self,+ llm_client: BaseChatModel,+ embedder: Embeddings,+ chunk_size: int = 2000,+ chunk_overlap: int = 200,+ max_workers: int = 10,+ verbose: bool = True,+ ):+ """+ Initialize Hyper_RAG engine.++ Args:+ llm_client (BaseChatModel): LLM client for text generation.+ embedder (Embeddings): Embedding model for text embeddings.+ chunk_size (int): Size of text chunks for indexing.+ chunk_overlap (int): Overlap between text chunks for indexing.+ max_workers (int): Maximum number of workers for indexing.+ verbose (bool): Display detailed execution logs and progress information.+ """+ # 1. Define Key Extractors (critical for deduplication)+ # Node key: exact match by name+ node_key_fn = lambda x: x.name++ # Edge key: sorted tuple of participants (set-like behavior for hyperedges)+ edge_key_fn = lambda x: tuple(sorted(x.participants))++ # 2. Edge consistency check: tell AutoHypergraph which nodes this edge connects+ nodes_in_edge_fn = lambda x: x.participants++ # 3. Create custom Mergers with LLM.CUSTOM_RULE strategy+ # Node merger: merges entity descriptions using NODE_MERGE_TEMPLATE+ node_merger = CustomRuleMerger(+ key_extractor=node_key_fn,+ llm_client=llm_client,+ item_schema=NodeSchema,+ rule=NODE_MERGE_TEMPLATE,+ )++ # Edge merger: merges relationship descriptions using EDGE_MERGE_TEMPLATE+ edge_merger = CustomRuleMerger(+ key_extractor=edge_key_fn,+ llm_client=llm_client,+ item_schema=EdgeSchema,+ rule=EDGE_MERGE_TEMPLATE,+ )++ # 4. Call parent class initialization+ super().__init__(+ node_schema=NodeSchema,+ edge_schema=EdgeSchema,+ node_key_extractor=node_key_fn,+ edge_key_extractor=edge_key_fn,+ nodes_in_edge_extractor=nodes_in_edge_fn,+ llm_client=llm_client,+ embedder=embedder,+ # Enforce two-stage extraction (nodes first, then edges)+ extraction_mode="two_stage",+ # Inject customized prompts for extraction+ prompt_for_node_extraction=HYPER_RAG_NODE_EXTRACTION_PROMPT,+ prompt_for_edge_extraction=HYPER_RAG_EDGE_EXTRACTION_PROMPT,+ # Pass custom merger instances+ node_strategy_or_merger=node_merger,+ edge_strategy_or_merger=edge_merger,+ # Optimize indexing+ node_fields_for_index=["name", "type", "description"],+ edge_fields_for_index=["keywords", "description"],+ # Other parameters+ chunk_size=chunk_size,+ chunk_overlap=chunk_overlap,+ max_workers=max_workers,+ verbose=verbose,+ )diff --git a/hyperextract/hypergraphs/hyperrag.py b/hyperextract/hypergraphs/hyperrag.pydeleted file mode 100644index e69de29..0000000