Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
refactor: 重构示例代码和测试文件结构
yifanfeng977fa394810fb2
diff --git a/README.md b/README.mdindex fb5a000..fe1539a 100644--- a/README.md+++ b/README.md@@ -12,33 +12,55 @@ Transform documents into **knowledge abstracts** — with just one command.---+## ✨ Features++- 🔷 **Knowledge Structures** — Model, List, Set, Graph, Hypergraph, TemporalGraph, SpatialGraph, SpatioTemporalGraph+- 💻 **Interactive CLI** — Extract, build, and interact with knowledge abstracts+- ⚡ **Incremental Updates** — Continuously update knowledge abstracts with new questions++---+## ⚡ Quick Start+### Install with uv+```bash-pip install hyper-extract+uv pip install hyper-extract+```-he config init -k YOUR_API_KEY+### CLI Usage-he parse document.md -o ./knowledge_abstract/ -l zh+```bash+he config init -k YOUR_API_KEY+he parse document.md -o ./output/ -l zh+he search ./output/ "What are the key events?"+he feed ./output/ new_document.md+he show ./output/```-> 🔗 For detailed usage, see [CLI Guide](./hyperextract/cli/README.md)+<details>+<summary>🐍 Python API</summary>----+```python+from hyperextract import Template-## ✨ Key Features+ka = Template.create("finance/event_timeline")+result = ka.parse(annual_report_text)+# result.timeline = [Event("Q1 revenue", "2024-01"), ...]+```-| 🤖 8 Knowledge Types | 🌍 38+ Domain Templates | 💻 Interactive CLI | 🔍 Semantic Search |-|:---:|:---:|:---:|:---:|-| From lists to spatio-temporal graphs | Finance, Medical, Legal ready-to-use | One command for full workflow | FAISS vector similarity |+</details>-> 🔗 Learn more about [8 Knowledge Types](#knowledge-types) | [Domain Templates](#domain-templates)+> 🔗 For detailed CLI usage, see [CLI Guide](./hyperextract/cli/README.md)----## 📊 Knowledge Types+## 📖 Knowledge Abstraction++> From simple structured data to complex spatio-temporal graphs — **10+ extraction methods**, covering **6 domains**, with **80+ templates**.-+<details>+<summary>🔧 8 Knowledge Structures</summary>| Type | Best For | Example ||------|----------|---------|@@ -51,13 +73,29 @@ he parse document.md -o ./knowledge_abstract/ -l zh| AutoSpatialGraph | Spatial relations | Delivery routes || AutoSpatioTemporalGraph | Events in time & space | Historical battles |-> 🔗 See [Full Type Documentation](./hyperextract/types/) | [Template Design Guide](./hyperextract/templates/DESIGN_GUIDE.md)+</details>----+<details>+<summary>🔍 Extraction Methods (10+)</summary>++| Method | Type | Description |+|--------|------|-------------|+| graph_rag | graph | Graph-RAG + community detection |+| light_rag | graph | Lightweight entity-relation extraction |+| hyper_rag | hypergraph | Hypergraph for multi-entity relations |+| cog_rag | graph | Cognitive retrieval augmentation |+| itext2kg | graph | High-quality triple extraction |+| kg_gen | graph | Structured knowledge generation |+| atom | graph | Temporal graph + evidence attribution |-## 🌍 Domain Templates+</details>-+<details>+<summary>🌍 Domain Templates (6 Domains / 38+ Templates)</summary>++Templates are written in YAML to define extraction targets and output structure.+- **Design Guide**: [Template Design Guide](./hyperextract/templates/DESIGN_GUIDE.md)+- **Preset Templates**: [presets directory](./hyperextract/templates/presets/)| Domain | Templates | Typical Scenarios ||--------|-----------|-------------------|@@ -68,21 +106,7 @@ he parse document.md -o ./knowledge_abstract/ -l zh| Industry | 5 | Equipment topology, Failure analysis || Legal | 5 | Contract clauses, Case citations |-> 🔗 See [Full Template Gallery](./hyperextract/templates/) | [Template Design Guide](./hyperextract/templates/DESIGN_GUIDE.md)-------## 🎯 Python API Examples--```python-from hyperextract import Template--ka = Template.create("finance/earnings_summary")-result = ka.parse(financial_report_text)-# result.operating_revenue = "100 billion RMB"-```--> 🔗 See more [Example Code](./examples/)+</details>---@@ -101,30 +125,14 @@ result = ka.parse(financial_report_text)----## 💻 CLI Reference--### Parse Documents--```bash-he parse document.md -o ./knowledge_abstract/ -l zh-he parse earnings.md -o ./finance_report/ -t finance/earnings_summary -l zh-```--### Search & Chat--```bash-he search ./finance_report/ "What was the revenue growth?"-he talk ./finance_report/ -i # Interactive mode-```--### Visualize--```bash-he show ./finance_report/ # Graph visualization-he info ./finance_report/ # Show statistics-```+## 📚 Related Documentation-> 🔗 For full CLI documentation, see [CLI Guide](./hyperextract/cli/README.md)+| Documentation | Description |+|---------------|-------------|+| [CLI Guide](./hyperextract/cli/README.md) | Complete CLI reference |+| [Template Gallery](./hyperextract/templates/) | 38+ domain templates |+| [Example Code](./examples/) | Python API examples |+| [Full Documentation](./docs/) | Architecture & implementation |---@@ -135,14 +143,3 @@ Contributions are welcome! Please submit Issues and PRs.## 📄 LicenseApache-2.0-------## 📚 Related Documentation--| Documentation | Description |-|---------------|-------------|-| [CLI Guide](./hyperextract/cli/README.md) | Complete CLI reference |-| [Template Gallery](./hyperextract/templates/) | 38+ domain templates |-| [Example Code](./examples/) | Python API examples |-| [Full Documentation](./docs/) | Architecture & implementation |diff --git a/README_ZH.md b/README_ZH.mdindex f4cd1ee..e1442e0 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -12,33 +12,55 @@---+## ✨ 核心特性++- 🔷 **知识结构** — Model, List, Set, Graph, Hypergraph, TemporalGraph, SpatialGraph, SpatioTemporalGraph+- 💻 **交互式CLI** — 提取、构建、与知识摘要交互+- ⚡ **增量更新** — 持续使用新问题更新知识摘要++---+## ⚡ 快速开始+### 使用 uv 安装+```bash-pip install hyper-extract+uv pip install hyper-extract+```-he config init -k YOUR_API_KEY+### CLI 使用-he parse document.md -o ./knowledge_abstract/ -l zh+```bash+he config init -k YOUR_API_KEY+he parse document.md -o ./output/ -l zh+he search ./output/ "有哪些关键事件?"+he feed ./output/ new_document.md+he show ./output/```-> 🔗 详细使用说明请查看 [CLI 指南](./hyperextract/cli/README.md)+<details>+<summary>🐍 Python API</summary>----+```python+from hyperextract import Template-## ✨ 核心特性+ka = Template.create("finance/event_timeline")+result = ka.parse(annual_report_text)+# result.timeline = [Event("Q1营收", "2024-01"), ...]+```-| 🤖 8种知识类型 | 🌍 38+领域模板 | 💻 交互式CLI | 🔍 语义检索 |-|:---:|:---:|:---:|:---:|-| 从列表到时空图谱 | 金融医疗法律开箱即用 | 一行命令完成全流程 | FAISS向量相似度 |+</details>-> 🔗 了解 [8种知识类型](#知识类型) | [领域模板](#领域模板)+> 🔗 详细 CLI 使用请查看 [CLI 指南](./hyperextract/cli/README.md)----## 📊 知识类型+## 📖 知识摘要-+> 从简单结构化到复杂时空图谱 —— **10+ 提取方法**,覆盖 **6 大领域**,共 **38+ 模板**。++<details>+<summary>🔧 8 种知识结构</summary>| 类型 | 适用场景 | 示例 ||------|----------|------|@@ -51,13 +73,30 @@ he parse document.md -o ./knowledge_abstract/ -l zh| AutoSpatialGraph | 空间关系 | 配送路线 || AutoSpatioTemporalGraph | 时空事件 | 历史战役 |-> 🔗 查看 [完整类型文档](./hyperextract/types/) | [模板设计指南](./hyperextract/templates/DESIGN_GUIDE.md)+</details>-----## 🌍 领域模板+<details>+<summary>🔍 提取方法 (10+)</summary>++| 方法 | 类型 | 描述 |+|------|------|------|+| graph_rag | graph | Graph-RAG + 社区检测 |+| light_rag | graph | 轻量级实体关系提取 |+| hyper_rag | hypergraph | 超图多实体关系 |+| cog_rag | graph | 认知检索增强 |+| itext2kg | graph | 高质量三元组提取 |+| kg_gen | graph | 结构化知识生成 |+| atom | graph | 时序图谱 + 证据归因 |++</details>-+<details>+<summary>🌍 领域模板 (6领域 / 80+模板)</summary>++模板使用 YAML 编写,定义提取目标和输出结构。+- **编写指南**: [模板设计指南](./hyperextract/templates/DESIGN_GUIDE.md)+- **预设模板**: [presets 目录](./hyperextract/templates/presets/)| 领域 | 模板数 | 典型场景 ||------|--------|----------|@@ -68,21 +107,7 @@ he parse document.md -o ./knowledge_abstract/ -l zh| Industry | 5 | 设备拓扑、事故分析 || Legal | 5 | 合同条款、判例引用 |-> 🔗 查看 [完整模板库](./hyperextract/templates/) | [模板设计指南](./hyperextract/templates/DESIGN_GUIDE.md)-------## 🎯 Python API 示例--```python-from hyperextract import Template--ka = Template.create("finance/earnings_summary")-result = ka.parse(financial_report_text)-# result.operating_revenue = "1000亿元"-```--> 🔗 查看更多 [示例代码](./examples/)+</details>---@@ -101,30 +126,14 @@ result = ka.parse(financial_report_text)----## 💻 CLI 参考--### 解析文档--```bash-he parse document.md -o ./knowledge_abstract/ -l zh-he parse earnings.md -o ./finance_report/ -t finance/earnings_summary -l zh-```--### 搜索与问答--```bash-he search ./finance_report/ "营收增长了多少?"-he talk ./finance_report/ -i # 交互式问答-```--### 可视化--```bash-he show ./finance_report/ # 图形化展示-he info ./finance_report/ # 显示统计信息-```+## 📚 相关文档-> 🔗 完整的 CLI 文档请查看 [CLI 指南](./hyperextract/cli/README.md)+| 文档 | 说明 |+|------|------|+| [CLI 指南](./hyperextract/cli/README.md) | 命令行工具完整参考 |+| [模板库](./hyperextract/templates/) | 38+ 领域模板 |+| [示例代码](./examples/) | Python API 使用示例 |+| [完整文档](./docs/) | 架构设计与实现细节 |---@@ -135,14 +144,3 @@ he info ./finance_report/ # 显示统计信息## 📄 许可证Apache-2.0-------## 📚 相关文档--| 文档 | 说明 |-|------|------|-| [CLI 指南](./hyperextract/cli/README.md) | 命令行工具完整参考 |-| [模板库](./hyperextract/templates/) | 38+ 领域模板 |-| [示例代码](./examples/) | Python API 使用示例 |-| [完整文档](./docs/) | 架构设计与实现细节 |diff --git a/examples/en/autotypes/graph_demo.py b/examples/en/autotypes/graph_demo.pyindex 8f7d35f..b10052b 100644--- a/examples/en/autotypes/graph_demo.py+++ b/examples/en/autotypes/graph_demo.py@@ -8,38 +8,48 @@ Usage:python examples/en/autotypes/graph_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class Entity(BaseModel):"""Entity in the knowledge graph"""+name: str = Field(description="Entity name")- type: str = Field(description="Entity type: person/location/invention", default="person")+ type: str = Field(+ description="Entity type: person/location/invention", default="person"+ )class Relation(BaseModel):"""Relation between entities"""+source: str = Field(description="Source entity")target: str = Field(description="Target entity")type: str = Field(description="Relation type: employer/partner/rival/invented")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" Graph Extraction Demo")print("=" * 60)@@ -48,6 +58,9 @@ def main():graph = AutoGraph[Entity, Relation](node_schema=Entity,edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),llm_client=llm,embedder=embedder,)@@ -58,16 +71,16 @@ def main():graph.build_index()- for q in [- "What are Tesla's major inventions?",- "What was the War of Currents?",- "Tesla's relationship with Edison",- ]:- print(f"\nQuery: {q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" Entities: {[n.name for n in nodes]}")- print(f" Relations: {[e.type for e in edges]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = graph.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: f"{x.type}")+\ No newline at end of filediff --git a/examples/en/autotypes/hypergraph_demo.py b/examples/en/autotypes/hypergraph_demo.pyindex d02dde3..e6657b9 100644--- a/examples/en/autotypes/hypergraph_demo.py+++ b/examples/en/autotypes/hypergraph_demo.py@@ -1,22 +1,27 @@"""Hypergraph Demo - Tesla Biography-Extract hyper-relationships from text using AutoHyperGraph.+Extract hyper-relationships from text using AutoHypergraph.This demo shows how to capture multi-entity relationships.Usage:python examples/en/autotypes/hypergraph_demo.py"""-import sysfrom pathlib import Path++from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv+from pydantic import BaseModel, Field-dotenv.load_dotenv()+from hyperextract.types import AutoHypergraph-from pydantic import BaseModel, Field-from hyperextract.types import AutoHyperGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"class Entity(BaseModel):@@ -32,22 +37,26 @@ class HyperEdge(BaseModel):type: str = Field(description="Relationship type")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" Hypergraph Demo")print("=" * 60)print("Extracting multi-entity relationships...")- graph = AutoHyperGraph[Entity, HyperEdge](+ graph = AutoHypergraph[Entity, HyperEdge](node_schema=Entity,edge_schema=HyperEdge,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.type}_{'_'.join(sorted(x.members))}",+ nodes_in_edge_extractor=lambda x: tuple(x.members),llm_client=llm,embedder=embedder,)@@ -62,11 +71,15 @@ def main():graph.build_index()- for q in ["War of Currents", "inventions", "collaboration"]:- print(f"\nQuery: {q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" Found {len(edges)} related hyper-edges")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = graph.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: x.type)diff --git a/examples/en/autotypes/list_demo.py b/examples/en/autotypes/list_demo.pyindex 2dbfdc9..34ff7e3 100644--- a/examples/en/autotypes/list_demo.py+++ b/examples/en/autotypes/list_demo.py@@ -8,16 +8,21 @@ Usage:python examples/en/autotypes/list_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract import AutoList+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class TimelineEvent(BaseModel):"""Timeline event"""@@ -26,14 +31,15 @@ class TimelineEvent(BaseModel):description: str = Field(description="Event description")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoList Demo - Tesla Timeline")print("=" * 60)@@ -55,12 +61,15 @@ def main():timeline.build_index()- for q in ["inventions", "Edison", "Colorado Springs"]:- print(f"\nQuery: {q}")- results = timeline.search(q, top_k=3)- for r in results:- print(f" {r.year}: {r.title}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = timeline.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ timeline.show(item_label_extractor=lambda x: f"{x.year}: {x.title}")diff --git a/examples/en/autotypes/model_demo.py b/examples/en/autotypes/model_demo.pyindex 7fcda3c..c7876ae 100644--- a/examples/en/autotypes/model_demo.py+++ b/examples/en/autotypes/model_demo.py@@ -8,17 +8,22 @@ Usage:python examples/en/autotypes/model_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from typing import List, Optional+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field-from typing import List, Optional+from hyperextract import AutoModel+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class BiographySummary(BaseModel):"""Biography summary schema"""@@ -32,14 +37,15 @@ class BiographySummary(BaseModel):major_inventions: List[str] = Field(default_factory=list, description="Major inventions")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoModel Demo - Tesla Summary")print("=" * 60)@@ -66,12 +72,15 @@ def main():model.build_index()- for q in ["inventions", "Edison", "legacy"]:- print(f"\nQuery: {q}")- results = model.search(q, top_k=2)- for r in results:- print(f" {r}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = model.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ model.show(label_extractor=lambda x: x.title)diff --git a/examples/en/autotypes/set_demo.py b/examples/en/autotypes/set_demo.pyindex 3102988..65329b8 100644--- a/examples/en/autotypes/set_demo.py+++ b/examples/en/autotypes/set_demo.py@@ -8,16 +8,21 @@ Usage:python examples/en/autotypes/set_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract import AutoSet+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class Entity(BaseModel):"""Entity with unique key"""@@ -26,14 +31,15 @@ class Entity(BaseModel):description: str = Field(description="Brief description", default="")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoSet Demo - Tesla Entities")print("=" * 60)@@ -61,12 +67,15 @@ def main():entities.build_index()- for q in ["Edison", "inventions", "Westinghouse"]:- print(f"\nQuery: {q}")- results = entities.search(q, top_k=3)- for r in results:- print(f" {r.name} ({r.category})")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = entities.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ entities.show(item_label_extractor=lambda x: x.name)diff --git a/examples/en/autotypes/spatial_demo.py b/examples/en/autotypes/spatial_demo.pyindex 9d39b9e..3196f19 100644--- a/examples/en/autotypes/spatial_demo.py+++ b/examples/en/autotypes/spatial_demo.py@@ -8,16 +8,21 @@ Usage:python examples/en/autotypes/spatial_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoSpatialGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class Location(BaseModel):"""Location entity"""@@ -33,14 +38,15 @@ class SpatialRelation(BaseModel):relation: str = Field(description="Spatial relationship type")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" Spatial Graph Demo")print("=" * 60)@@ -49,6 +55,10 @@ def main():graph = AutoSpatialGraph[Location, SpatialRelation](node_schema=Location,edge_schema=SpatialRelation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ location_in_edge_extractor=lambda x: x.target,llm_client=llm,embedder=embedder,observation_location="United States",@@ -63,11 +73,15 @@ def main():graph.build_index()- for q in ["New York", "Colorado Springs", "labs"]:- print(f"\nQuery: {q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" Locations: {[n.name for n in nodes]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = graph.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: x.relation)diff --git a/examples/en/autotypes/spatio_temporal_demo.py b/examples/en/autotypes/spatio_temporal_demo.pyindex 1c49f0b..0f8e2ef 100644--- a/examples/en/autotypes/spatio_temporal_demo.py+++ b/examples/en/autotypes/spatio_temporal_demo.py@@ -8,17 +8,22 @@ Usage:python examples/en/autotypes/spatio_temporal_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from typing import Optional+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field-from typing import Optional+from hyperextract.types import AutoSpatioTemporalGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class Entity(BaseModel):"""Entity node"""@@ -35,14 +40,15 @@ class SpatioTemporalEvent(BaseModel):location: Optional[str] = Field(description="Where", default=None)-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" Spatio-Temporal Graph Demo")print("=" * 60)@@ -51,6 +57,11 @@ def main():graph = AutoSpatioTemporalGraph[Entity, SpatioTemporalEvent](node_schema=Entity,edge_schema=SpatioTemporalEvent,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.action}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ time_in_edge_extractor=lambda x: x.time or "",+ location_in_edge_extractor=lambda x: x.location or "",llm_client=llm,embedder=embedder,observation_location="United States",@@ -68,11 +79,15 @@ def main():graph.build_index()- for q in ["New York", "Colorado Springs", "inventions"]:- print(f"\nQuery: {q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" Found {len(nodes)} entities, {len(edges)} events")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = graph.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: f"{x.action}@{x.time}")diff --git a/examples/en/autotypes/temporal_demo.py b/examples/en/autotypes/temporal_demo.pyindex a03f65e..1a38cd4 100644--- a/examples/en/autotypes/temporal_demo.py+++ b/examples/en/autotypes/temporal_demo.py@@ -8,16 +8,21 @@ Usage:python examples/en/autotypes/temporal_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoTemporalGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"+class Entity(BaseModel):"""Entity node"""@@ -32,14 +37,15 @@ class TimelineEvent(BaseModel):entities: list[str] = Field(description="Entities involved")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "en" / "tesla.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" Temporal Graph Demo")print("=" * 60)@@ -48,6 +54,10 @@ def main():graph = AutoTemporalGraph[Entity, TimelineEvent](node_schema=Entity,edge_schema=TimelineEvent,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.time}-{x.event}",+ nodes_in_edge_extractor=lambda x: tuple(x.entities) if x.entities else (),+ time_in_edge_extractor=lambda x: x.time,llm_client=llm,embedder=embedder,)@@ -62,11 +72,15 @@ def main():graph.build_index()- for q in ["1884", "Colorado Springs", "Wardenclyffe"]:- print(f"\nQuery: {q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" Events: {[e.time for e in edges]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("Q&A")+ print("-" * 60)+ for q in questions:+ print(f"\nQ: {q}")+ try:+ result = graph.chat(q)+ print(f"A: {result.content}")+ except Exception as e:+ print(f"Error: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: f"{x.event}@{x.time}")diff --git a/examples/en/methods/atom_demo.py b/examples/en/methods/atom_demo.pyindex e8d936b..aa42dd6 100644--- a/examples/en/methods/atom_demo.py+++ b/examples/en/methods/atom_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/atom_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import Atom+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/cog_rag_demo.py b/examples/en/methods/cog_rag_demo.pyindex a905ac2..67b11f3 100644--- a/examples/en/methods/cog_rag_demo.py+++ b/examples/en/methods/cog_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/cog_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Cog_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:@@ -37,7 +37,7 @@ if __name__ == "__main__":rag = Cog_RAG(llm_client=llm, embedder=embedder)rag.feed_text(text)- print(f"\n✓ Extracted {len(rag.themes)} themes, {len(rag.nodes)} entities\n")+ print(f"\n✓ Extracted {len(rag.nodes)} entities, {len(rag.edges)} hyperedges\n")print("-" * 60)print("Q&A")diff --git a/examples/en/methods/graph_rag_demo.py b/examples/en/methods/graph_rag_demo.pyindex 8bcda98..a7ba941 100644--- a/examples/en/methods/graph_rag_demo.py+++ b/examples/en/methods/graph_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/graph_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Graph_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/hyper_rag_demo.py b/examples/en/methods/hyper_rag_demo.pyindex f4a6b01..52f9f3d 100644--- a/examples/en/methods/hyper_rag_demo.py+++ b/examples/en/methods/hyper_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/hyper_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Hyper_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/hypergraph_rag_demo.py b/examples/en/methods/hypergraph_rag_demo.pyindex aff79df..a7635fc 100644--- a/examples/en/methods/hypergraph_rag_demo.py+++ b/examples/en/methods/hypergraph_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/hypergraph_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import HyperGraph_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/itext2kg_demo.py b/examples/en/methods/itext2kg_demo.pyindex b1b8230..528df8e 100644--- a/examples/en/methods/itext2kg_demo.py+++ b/examples/en/methods/itext2kg_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/itext2kg_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import iText2KG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/itext2kg_star_demo.py b/examples/en/methods/itext2kg_star_demo.pyindex 2cb96cf..5894906 100644--- a/examples/en/methods/itext2kg_star_demo.py+++ b/examples/en/methods/itext2kg_star_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/itext2kg_star_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import iText2KG_Star+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/kg_gen_demo.py b/examples/en/methods/kg_gen_demo.pyindex db955a5..8cfd68c 100644--- a/examples/en/methods/kg_gen_demo.py+++ b/examples/en/methods/kg_gen_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/kg_gen_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import KG_Gen+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/methods/light_rag_demo.py b/examples/en/methods/light_rag_demo.pyindex 6ce96ab..bc7227e 100644--- a/examples/en/methods/light_rag_demo.py+++ b/examples/en/methods/light_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/en/methods/light_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Light_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "en" / "tesla.md"-QUESTION_FILE = project_root / "examples" / "en" / "tesla_question.md"+INPUT_FILE = project_root / "en" / "tesla.md"+QUESTION_FILE = project_root / "en" / "tesla_question.md"if __name__ == "__main__":with open(INPUT_FILE) as f:diff --git a/examples/en/templates/finance_template.py b/examples/en/templates/finance_template.pyindex 19e147b..c37be13 100644--- a/examples/en/templates/finance_template.py+++ b/examples/en/templates/finance_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/finance_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/templates/general_template.py b/examples/en/templates/general_template.pyindex 259447d..2acd851 100644--- a/examples/en/templates/general_template.py+++ b/examples/en/templates/general_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/general_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/templates/industry_template.py b/examples/en/templates/industry_template.pyindex 00df6d8..ea438e6 100644--- a/examples/en/templates/industry_template.py+++ b/examples/en/templates/industry_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/industry_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/templates/legal_template.py b/examples/en/templates/legal_template.pyindex d23bf00..4503238 100644--- a/examples/en/templates/legal_template.py+++ b/examples/en/templates/legal_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/legal_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/templates/list_templates.py b/examples/en/templates/list_templates.pyindex 8e8a452..e1f2c90 100644--- a/examples/en/templates/list_templates.py+++ b/examples/en/templates/list_templates.py@@ -5,12 +5,10 @@ Usage:python examples/en/templates/list_templates.py"""-import sysimport yamlfrom pathlib import Pathproject_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))def list_templates():diff --git a/examples/en/templates/medicine_template.py b/examples/en/templates/medicine_template.pyindex 2804fa8..a675a0d 100644--- a/examples/en/templates/medicine_template.py+++ b/examples/en/templates/medicine_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/medicine_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/templates/tcm_template.py b/examples/en/templates/tcm_template.pyindex 321ce9c..efd4609 100644--- a/examples/en/templates/tcm_template.py+++ b/examples/en/templates/tcm_template.py@@ -5,16 +5,14 @@ Usage:python examples/en/templates/tcm_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/en/tesla_question.md b/examples/en/tesla_question.mdindex 01568ff..0fdcf9c 100644--- a/examples/en/tesla_question.md+++ b/examples/en/tesla_question.md@@ -1,5 +1,3 @@-1. What are Tesla's major inventions and their significance?-2. What was the "War of Currents" and who were the main participants?-3. How did Tesla's relationship with Edison evolve over time?-4. What is Tesla's relationship with Westinghouse?-5. What happened at Wardenclyffe Tower?+- What are Tesla's major inventions and their significance?+- What was the "War of Currents" and who were the main participants?+- How did Tesla's relationship with Edison evolve over time?\ No newline at end of filediff --git a/examples/types/graph_demo.py b/examples/types/graph_demo.pydeleted file mode 100644index 23846e6..0000000--- a/examples/types/graph_demo.py+++ /dev/null@@ -1,249 +0,0 @@-"""-AutoGraph Demo: MMORPG Guild Drama (游戏公会恩仇录) 🎮⚔️--场景背景:-在一个名为《永恒之境》的游戏中,顶尖公会"星辰阁"在攻略最终BOSS时发生了严重的内讧。-这段文本是一篇游戏论坛的"818"(八卦爆料)帖子。--难点:-1. 术语混杂:ID、职业代称(主T、奶妈)、外号混用。-2. 关系复杂:涉及治疗、仇恨(Aggro)、装备分配(Loot)、公会管理。-3. 讽刺与隐喻:LLM 需要理解"划水"、"黑装备"等游戏黑话。--使用方法:- python examples/types/graph_demo.py-"""--import sys-from pathlib import Path--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from langchain_openai import ChatOpenAI, OpenAIEmbeddings--from hyperextract.types import AutoGraph-from ontomem import MergeStrategy--import dotenv--dotenv.load_dotenv()--INPUT_FILE = project_root / "examples" / "inputs" / "mmorpg_guild_drama.md"--# ==================== 1. 定义游戏知识结构 (Schema) ====================---class GameEntity(BaseModel):- """游戏实体节点(包含玩家、BOSS、NPC、公会等)"""-- name: str = Field(description="实体名称或ID,例如玩家ID、BOSS名、'星辰阁'")- category: str = Field(- description="实体类型,例如:'玩家', 'BOSS', '公会', '群体'", default="玩家"- )- info: str = Field(- description="职业、描述或状态,如'战神'、'最终BOSS'、'解散'", default="未知"- )-- def __repr__(self):- return f"🎮 [{self.category}] {self.name} <{self.info}>"---class GameInteraction(BaseModel):- """游戏互动边"""-- source: str = Field(description="发起互动的实体名称")- target: str = Field(description="互动的对象名称(可以是玩家、BOSS或公会)")- action_type: str = Field(- description="互动类型:'治疗', '攻击', '抢装备', '辱骂', '踢出队伍', '拉入队伍'"- )- details: str = Field(description="具体的互动描述或原因")-- def __repr__(self):- return (- f"⚡ {self.source} --[{self.action_type}]--> {self.target} ({self.details})"- )---# ==================== 2. 运行提取 ====================--def run_demo():- print("=" * 60)- print("🕹️ 正在加载游戏日志分析模块...")- print("=" * 60)-- # 读取输入文件- print(f"\n📖 正在阅读游戏论坛爆料帖: {INPUT_FILE}")- print("=" * 60)- with open(INPUT_FILE, "r", encoding="utf-8") as f:- game_forum_post = f.read()- print(game_forum_post[:200] + "...")- print("=" * 60)-- # 初始化 LLM- print("\n🔧 初始化 LLM 和嵌入模型...")- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings()-- # 创建 AutoGraph- print("📊 构建 AutoGraph 实例...\n")- graph = AutoGraph[GameEntity, GameInteraction](- node_schema=GameEntity,- edge_schema=GameInteraction,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}_{x.action_type}_{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder,- extraction_mode="two_stage",- node_strategy_or_merger=MergeStrategy.LLM.BALANCED,- prompt_for_node_extraction=(- "提取文本中的所有游戏实体作为节点。关键:必须提取BOSS(如深渊魔龙)、公会(如星辰阁)以及关键玩家。"- "如果文中提到'全团'或'所有人',也将其作为一个群体实体提取。"- "注意识别ID和别名,将别名(如'团长')标准化为具体ID。\n\n"- "## 源文本:\n"- "{source_text}"- ),- prompt_for_edge_extraction=(- "提取实体之间的互动行为。"- "重要规则:互动中的 source 和 target 必须是你刚才提取过的实体名称。"- "例如,如果之前提取了节点'深渊魔龙',那么攻击BOSS的互动 target 必须填'深渊魔龙'。"- "### 已知的实体列表\n"- "{known_nodes}\n\n"- "## 源文本:\n"- "{source_text}\n\n"- ),- )-- # 执行提取- print("\n⚙️ 执行两阶段提取...\n")- graph.feed_text(game_forum_post)-- # ==================== 4. 战报分析 ====================-- print("\n" + "=" * 60)- print(f"✅ 分析完成!")- print("=" * 60)- print(f" 找到玩家/BOSS: {len(graph.nodes)} 个")- print(f" 记录的互动事件: {len(graph.edges)} 个")-- print("\n" + "-" * 60)- print("🛡️ 角色面板 (Nodes)")- print("-" * 60)- for i, p in enumerate(graph.nodes, 1):- print(f"{i}. {p}")-- print("\n" + "-" * 60)- print("⚔️ 战斗和互动记录 (Edges)")- print("-" * 60)- for i, e in enumerate(graph.edges, 1):- print(f"{i}. {e}")-- # ==================== 5. 语义搜索 (查内鬼) ====================-- print("\n" + "=" * 60)- print("🔍 游戏管理员(GM)调查查询模块")- print("=" * 60)-- # 建立索引(默认同时索引节点和边,无需指定参数)- graph.build_index()-- # 场景1: 找谁拿了装备?- q1 = "谁拿走了传说武器?"- print(f"\n❓ Query: {q1}")- try:- # 默认同时搜索节点和边,返回 Tuple[nodes, edges]- nodes, edges = graph.search(q1, top_k_nodes=3, top_k_edges=3)- interactions = edges- if interactions:- for i in interactions:- print(f" 🚨 {i}")- else:- print(" (没有找到相关记录)")- except Exception as e:- print(f" (搜索出错: {e})")-- # 场景2: 找特定的职业- q2 = "谁是治疗职业的?"- print(f"\n❓ Query: {q2}")- try:- nodes, edges = graph.search(q2, top_k_nodes=3, top_k_edges=3)- healers = nodes- if healers:- for h in healers:- print(f" 🚑 {h}")- else:- print(" (没有找到治疗职业)")- except Exception as e:- print(f" (搜索出错: {e})")-- # 场景3: 找矛盾中心- q3 = "谁和谁发生了冲突?"- print(f"\n❓ Query: {q3}")- try:- nodes, edges = graph.search(q3, top_k_nodes=3, top_k_edges=3)- conflicts = edges- if conflicts:- for c in conflicts:- print(f" 🔥 {c}")- else:- print(" (没有找到冲突记录)")- except Exception as e:- print(f" (搜索出错: {e})")-- # 场景4: 找坦克- q4 = "谁是主坦克?"- print(f"\n❓ Query: {q4}")- try:- nodes, edges = graph.search(q4, top_k_nodes=3, top_k_edges=3)- tanks = nodes- if tanks:- for t in tanks:- print(f" 🛡️ {t}")- else:- print(" (没有找到坦克)")- except Exception as e:- print(f" (搜索出错: {e})")-- # ==================== 6. 交互式对话 (Chat RAG) ====================-- print("\n" + "=" * 60)- print("💬 进入交互式对话模式")- print("=" * 60)- print("🤖 正在基于提取的知识图谱回答复杂问题...\n")-- chat_queries = [- "请总结这个公会冲突的核心原因。",- "暗影刺客和小甜甜的行为对团队造成了什么影响?",- "从这个事件中,我们学到了什么关于公会管理的经验?",- ]-- for q in chat_queries:- print(f"👤 提问: {q}")- try:- response = graph.chat(q)- print(f"🤖 回答: {response.content}\n")- except Exception as e:- print(f"⚠️ 对话异常: {e}\n")-- # ==================== 7. 保存知识图谱 ====================-- print("\n" + "=" * 60)- print("💾 保存知识图谱到本地...")- print("=" * 60)- save_path = project_root / "temp" / "auto_graph_demo"- try:- graph.dump(save_path)- print(f"✅ 图谱已保存到: {save_path}")- except Exception as e:- print(f"⚠️ 保存失败: {e}")-- graph.show(- node_label_extractor=lambda n: f"{n.name} ({n.category})",- edge_label_extractor=lambda e: e.action_type,- )---if __name__ == "__main__":- run_demo()diff --git a/examples/types/hypergraph_demo.py b/examples/types/hypergraph_demo.pydeleted file mode 100644index 1340edc..0000000--- a/examples/types/hypergraph_demo.py+++ /dev/null@@ -1,389 +0,0 @@-"""-AutoHypergraph 演示:星际外交危机的故事线分析--这个例子展示了如何使用 AutoHypergraph 从复杂的叙事文本中提取多类型的超边关系。-超边可以代表不同类型的故事单元:冲突、结盟、持有关系、会议等。--Usage:- python examples/types/hypergraph_demo.py-"""--import os-import sys-import collections-from typing import List-from pathlib import Path--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from dotenv import load_dotenv-from pydantic import BaseModel, Field-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-from hyperextract.types import AutoHypergraph--load_dotenv()--INPUT_FILE = project_root / "examples" / "inputs" / "sci_fi_story.md"--# ==============================================================================-# 1. 定义 Schema:灵活的叙事结构-# ==============================================================================---class StoryEntity(BaseModel):- """- 节点:故事中的基本元素。- 可以是:人物(Character), 地点(Location), 物品(Item), 派系(Faction)- """-- name: str = Field(description="实体名称,如 'Neo', 'Matrix', 'Excalibur'")- category: str = Field(- description="实体类别: Character(人物), Location(地点), Item(物品), Faction(派系)",- )- description: str = Field(- description="实体的详细描述,如 '黑蛇'(Character) 的角色是 Which Snake",- )---class NarrativeUnit(BaseModel):- """- 超边:一个叙事单元。-- 这不仅仅是一个关系,它可以是一个发生的事件、一种复杂的联盟、或者一个状态。- 关键在于:它通过一个 'type' 字段来区分语义,但结构上都是连接多个实体的超边。- """-- summary: str = Field(description="简短描述这个单元发生了什么,或者是什么关系")-- # 这里体现了"多类型":LLM 会判断这具体是什么类型的连接- edge_type: str = Field(description="超边类型: Conflict(冲突), Alliance(结盟), Possession(持有), Meeting(会议), State(状态)")-- # 核心:连接所有相关的实体- participants: List[str] = Field(description="所有牵涉其中的实体名称列表")---# ==============================================================================-# 2. 故事文本:星际外交危机-# ==============================================================================--source_text = """-【银河纪元 3042年 - 赤红星云事件报告】--危机在这一周达到了顶峰。整个银河系似乎都陷入了暗流涌动的政治漩涡。--【第一阶段:秘密接触】-臭名昭著的星际海盗黑蛇驾驶着他的旗舰复仇号,在泰坦空间站秘密会见了银河联邦的背叛者议员 Valerius。-目击者称,两人在午夜时分登上了空间站的顶层禁区。据情报部门推测,这是一次肮脏的交易——-双方达成了针对皇室的同盟关系。参与此会议的还有 Valerius 手下的特务头目 Shadow,-他负责处理所有不可告人的细节。这个同盟的形成标志着大麻烦即将到来。--【第二阶段:叛变与夺权】-在这次会议的两天后,事态急剧恶化。黑蛇这种疯狂的家伙竟然在极光星区伏击了皇家护卫队。-这场突然的冲突中,皇家护卫队的队长 Leona 与副队长 Marcus 奋力抵抗,-但他们陷入了黑蛇手下的精锐战士的包围。这场激战导致队长 Leona 身受重伤。-而黑蛇的目标不是为了杀戮,而是为了夺取一件神圣的物品——传说中的虚空水晶。-这枚水晶散发着幽蓝的光芒,据说能够打开先祖之门的第一道密封。-黑蛇在战斗中成功夺取了水晶,但 Leona 在临终前用最后的力量将其重新夺回。--【第三阶段:流亡与隐藏】-Leona 被迫带着虚空水晶,从极光星区逃往荒凉的废土行星 Z-9。-在这个被遗忘的角落,她向仅有的几个信任者求救:还有副队长 Marcus 以及隐秘组织"地下抵抗军"的领导人 Cipher。-同时,议员 Valerius 非法持有了开启先祖之门的钥匙——这把钥匙本应被安全保管在皇家金库中。-他现在藏匿在浮空都市中,与黑蛇遥相呼应,计划着下一步的政变。--【第四阶段:势力重组】-银河联邦的最高委员会得知了这一系列事件,决定采取行动。-高级顾问 Artemis 被任命为特使,她与 Leona 在废土行星 Z-9 进行了秘密会晤,-讨论如何制止这场即将到来的危机。同时,地下抵抗军的 Cipher 也加入了这次会议,-三人共同制定了一个大胆的计划。--【第五阶段:最后的筹码】-虽然黑蛇未能获得虚空水晶,但他已经从 Valerius 那里获得了先祖之门钥匙的复制品。-根据情报,这个复制品目前被秘密运往到黑蛇的秘密基地——位于辐射荒漠中的要塞。-参与此次运送的还有 Shadow 和黑蛇手下的精锐战士群体。-与此同时,Leona 和 Marcus 正在准备一场反击,他们计划在虚空水晶的力量下重新夺回局势的控制权。--目前局势:-- 黑蛇:手持复制品钥匙,掌控旗舰复仇号,计划进行最终的攻击-- 议员 Valerius:躲在浮空都市中,与黑蛇保持联系-- Leona:守护虚空水晶,在废土行星 Z-9 养伤,准备反击-- 银河联邦:通过特使 Artemis 支持 Leona,正在调集力量-- 地下抵抗军:Cipher 领导,与 Leona 和 Marcus 联手,准备对抗黑蛇和 Valerius 的同盟--这场战争的走向仍不确定。虚空水晶、先祖之门、以及隐藏在这一切背后的秘密,-都将在接下来的日子里浮出水面。银河的命运,可能就在这些关键人物和物品的碰撞中决定。-"""---# ==============================================================================-# 3. 提取逻辑配置-# ==============================================================================---def main():- """主函数:完整的 AutoHypergraph 提取与分析流程"""-- # 初始化 LLM 和嵌入模型- llm_client = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings()-- # 1. 节点指纹:用名字即可- node_key_fn = lambda x: x.name-- # 2. 边指纹:类型 + 摘要 + 参与者(排序)- # 这样即使是同一群人,如果发生了两件事,也会被视为两条边- def edge_key_fn(x: NarrativeUnit) -> str:- participants = sorted(x.participants)- return f"{x.edge_type}|{'-'.join(participants)}"-- # 3. 参与者映射:告诉框架谁在这个超边里- participants_fn = lambda x: tuple(x.participants)-- print("=" * 70)- print("🚀 AutoHypergraph 演示:赤红星云事件分析")- print("=" * 70)- print()-- # 创建 AutoHypergraph 实例- hypergraph = AutoHypergraph(- node_schema=StoryEntity,- edge_schema=NarrativeUnit,- node_key_extractor=node_key_fn,- edge_key_extractor=edge_key_fn,- nodes_in_edge_extractor=participants_fn,- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage", # 推荐:先识别实体,再梳理故事线- # 使用中文 Prompt 引导 LLM- prompt_for_node_extraction=(- "你的任务是从故事文本中提取所有重要的实体节点。\n"- "实体可以是以下类别之一:\n"- "- Character(人物):故事中出现的角色\n"- "- Location(地点):故事发生的地方\n"- "- Item(物品):故事中提到的重要物品或武器\n"- "- Faction(派系):故事中的组织或势力\n\n"- "请确保提取所有重要的实体,并正确分类它们的类别。\n\n"- "## 源文本:\n"- "{source_text}"- ),- prompt_for_edge_extraction=(- "你的任务是从故事文本中提取复杂的叙事关系,将其建模为超边。\n"- "超边可以连接任意数量的实体(Character、Location、Item、Faction)。\n\n"- "请根据故事内容,识别并分类以下类型的超边:\n"- "- Conflict(冲突):两个或多个角色之间的战斗或对抗\n"- "- Alliance(结盟):角色或派系之间的同盟或合作关系\n"- "- Possession(持有):某个角色拥有或守护某件物品\n"- "- Meeting(会议):多个角色在某个地点进行的会面\n"- "- State(状态):描述某个角色或物品在某个地点的状态\n\n"- "对于每个超边,请提供:\n"- "1. summary:对这个叙事单元的简洁描述\n"- "2. edge_type:上述分类之一\n"- "3. participants:所有牵涉其中的实体名称列表\n\n"- "确保 participants 列表包含故事中与该事件相关的所有实体。\n"- "### 已知的实体列表\n"- "{known_nodes}\n\n"- "## 源文本:\n"- "{source_text}\n\n"- ),- chunk_size=1000,- )-- print("📚 正在分析《赤红星云事件报告》...")- print(f" 文本长度:{len(source_text)} 字符")- print()-- # 执行提取- hypergraph.feed_text(source_text)-- # ==============================================================================- # 4. 结果展示:按类型分类- # ==============================================================================-- print("\n" + "=" * 70)- print(f"🎭 第一部分:登场角色与物品 ({len(hypergraph.nodes)} 个实体)")- print("=" * 70)-- # 按类别打印- nodes_by_cat = collections.defaultdict(list)- for n in hypergraph.nodes:- nodes_by_cat[n.category].append(n.name)-- category_emoji = {"Character": "👤", "Location": "📍", "Item": "⚔️", "Faction": "🏛️"}-- for cat in ["Character", "Location", "Item", "Faction"]:- if cat in nodes_by_cat:- emoji = category_emoji.get(cat, "•")- names = ", ".join(nodes_by_cat[cat])- print(- f"\n {emoji} {cat}({cat == 'Character' and '人物' or cat == 'Location' and '地点' or cat == 'Item' and '物品' or '派系'}):"- )- print(f" {names}")-- # ==============================================================================- # 5. 故事线展示:按超边类型分类- # ==============================================================================-- print("\n" + "=" * 70)- print(f"🎬 第二部分:故事线分析 ({len(hypergraph.edges)} 个叙事单元)")- print("=" * 70)-- # 按类型分组- edges_by_type = collections.defaultdict(list)- for e in hypergraph.edges:- edges_by_type[e.edge_type].append(e)-- type_emoji = {- "Conflict": "⚔️",- "Alliance": "🤝",- "Possession": "🎁",- "Meeting": "🤵",- "State": "📊",- }-- type_cn = {- "Conflict": "冲突",- "Alliance": "结盟",- "Possession": "持有",- "Meeting": "会议",- "State": "状态",- }-- for e_type in ["Conflict", "Alliance", "Meeting", "Possession", "State"]:- if e_type in edges_by_type:- emoji = type_emoji.get(e_type, "•")- cn_name = type_cn.get(e_type, e_type)- edges = edges_by_type[e_type]- print(f"\n {emoji} 【{cn_name}】类型 ({len(edges)} 个事件):")- for i, e in enumerate(edges, 1):- print(f"\n {i}. 事件描述:{e.summary}")- print(f" 🔗 参与方:{', '.join(e.participants)}")-- # ==============================================================================- # 6. 实体中心性分析:哪个实体参与了最多的故事线- # ==============================================================================-- print("\n" + "=" * 70)- print("📊 第三部分:关键实体分析(超边参与度)")- print("=" * 70)-- entity_participation = collections.defaultdict(int)- for e in hypergraph.edges:- for participant in e.participants:- entity_participation[participant] += 1-- # 按参与度排序- sorted_entities = sorted(- entity_participation.items(), key=lambda x: x[1], reverse=True- )-- print("\n 参与故事线最多的实体(中心性分析):\n")- for rank, (entity, count) in enumerate(sorted_entities[:10], 1):- bar = "█" * count- print(f" {rank:2d}. {entity:12s} {bar} ({count} 个故事线)")-- # ==============================================================================- # 7. 复杂查询演示- # ==============================================================================-- print("\n" + "=" * 70)- print("🔍 第四部分:语义搜索演示")- print("=" * 70)-- # 建立索引- print("\n 正在构建搜索索引...")- hypergraph.build_index(index_nodes=True, index_edges=True)-- # 执行几个查询- queries = [- "虚空水晶在哪里?谁守护着它?",- "Leona 发生了什么事?",- "黑蛇的计划是什么?",- ]-- for query in queries:- print(f"\n 🎯 查询:{query}")- results = hypergraph.search_edges(query, top_k=3)- if results:- for j, res in enumerate(results, 1):- print(f" → 结果 {j} [{res.edge_type}]:{res.summary}")- else:- print(f" → 未找到相关信息")-- # ==============================================================================- # 8. 关系图统计- # ==============================================================================-- print("\n" + "=" * 70)- print("📈 第五部分:图统计")- print("=" * 70)-- print(f"\n 节点总数(实体):{len(hypergraph.nodes)}")- print(f" 边总数(叙事单元):{len(hypergraph.edges)}")-- avg_participants = (- sum(len(e.participants) for e in hypergraph.edges) / len(hypergraph.edges)- if hypergraph.edges- else 0- )- print(f" 平均每条边的参与度:{avg_participants:.2f} 个实体")-- max_participants_edge = (- max(hypergraph.edges, key=lambda e: len(e.participants))- if hypergraph.edges- else None- )- if max_participants_edge:- print(f" 最复杂的叙事单元:{max_participants_edge.summary}")- print(f" 参与实体数:{len(max_participants_edge.participants)}")-- # ==============================================================================- # 9. 保存超图到本地- # ==============================================================================-- print("\n" + "=" * 70)- print("💾 保存超图到本地")- print("=" * 70)-- save_path = project_root / "temp" / "auto_hypergraph_demo"-- try:- hypergraph.dump(save_path)- print(f"\n ✅ 超图已保存到: {save_path}")- print(f" 节点数: {len(hypergraph.nodes)}")- print(f" 边数: {len(hypergraph.edges)}")- except Exception as e:- print(f"\n ⚠️ 保存失败: {e}")-- # ==================== 第六部分:智能对话 ====================- print("\n" + "=" * 70)- print("💬 超图智能问答")- print("=" * 70)- print("基于提取的超边关系进行多角度分析...\n")-- hypergraph_questions = [- "这个研究小组的主要合作特点是什么?",- "团队成员之间的互动模式如何?",- "从超图结构看,谁是这个社区的关键人物?"- ]-- for q in hypergraph_questions:- print(f"🔍 分析师: {q}")- try:- result = hypergraph.chat(q)- print(f"📊 深层分析: {result.content}\n")- except Exception as e:- print(f"⚠️ 分析异常: {e}\n")-- print("\n" + "=" * 70)- print("✅ 分析完成!")- print("=" * 70)-- hypergraph.show(- node_label_extractor=lambda n: f"{n.name}\n({n.category})",- edge_label_extractor=lambda e: f"{e.edge_type}",- )---if __name__ == "__main__":- main()diff --git a/examples/types/list_demo.py b/examples/types/list_demo.pydeleted file mode 100644index 6ded5fb..0000000--- a/examples/types/list_demo.py+++ /dev/null@@ -1,242 +0,0 @@-"""-AutoList 多 Chunk 提取示例--演示如何使用 AutoList 处理长文本,测试多 chunk 并发提取和列表合并功能。--Usage:- python examples/types/list_demo.py-"""--import sys-from pathlib import Path--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from typing import Optional-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-from hyperextract import AutoList--import dotenv--dotenv.load_dotenv()---# 定义单个新闻事件的 Schema-class NewsEvent(BaseModel):- """新闻事件结构"""-- title: str = Field(description="事件标题")- date: str = Field(description="事件发生日期")- location: str = Field(description="事件发生地点")- description: str = Field(description="事件描述")- participants: list[str] = Field(default_factory=list, description="参与者")- impact: str = Field(default="", description="事件影响")---def generate_long_news_text() -> str:- """生成包含多个新闻事件的长文本"""- return """-2024年的科技圈可谓是风起云涌,一个接一个的突破性进展改变着世界。这一切始于一月份,当时旧金山的全球目光都被吸引到了OpenAI的总部。-1月15日,OpenAI正式发布了备受期待的GPT-5大语言模型,这无疑是一个足以震撼业界的时刻。CEO Sam Altman在发布会上自信地宣布,新模型的参数量达到了-惊人的5万亿,训练数据量是GPT-4的整整10倍。这套堪称最强的语言模型在推理能力、多模态理解和长文本处理方面实现了重大突破。此次发布引起了全球AI-行业的广泛关注,纳斯达克上那些科技公司股价应声上涨,分析师纷纷调高评级。而微软、谷歌这些竞争对手一看这架势坐不住了,立即宣布将加速自家AI-产品的迭代。业界的共识是,这不仅仅是一个产品发布,而是标志着人工智能进入了一个全新的发展阶段。--仅仅一周后,又一个科技大事件在德克萨斯州的奥斯汀上演。特斯拉在1月22日正式发布了FSD(Full Self-Driving) V12版本,-Elon Musk亲自宣布这是第一个完全基于端到端神经网络的自动驾驶系统,彻底摒弃了传统的人工规则编码。这套新系统在城市道路、高速公路和复杂停车-场景中的表现都有了显著提升,而且已经有超过100万名测试用户参与了Beta测试并提交了反馈。美国交通部也很给面子,迅速批准了临时许可,预计将在-3月份向所有车主推送这个更新。整个业界都认为这是自动驾驶技术向商业化迈进的一个里程碑事件。--到了二月份,硬件领域也开始发力。2月1日,苹果在其加州库比蒂诺的Apple Park总部举行了盛大发布会,推出了第二代混合现实头显Vision Pro 2。-与第一代相比,这款新产品经过了精心的工业设计优化,重量足足减轻了30%,续航时间提升至8小时,售价也从天价的3499美元降至2499美元,虽然依然-不便宜但相对"亲民"了许多。CEO Tim Cook在舞台上演示了搭载的更强大M3芯片和革新性的眼动追踪技术的完美配合。首发市场覆盖美国、中国、日本-和欧洲主要国家,预购首日的订单量就突破了50万台,远远超过了第一代产品的预期。分析师普遍预测,2024年的MR头显市场将迎来一场爆发式增长。--就在Vision Pro 2发布两周后,量子计算领域传来了同样振奋人心的消息。2月14日,IBM在纽约约克城研究中心宣布成功研制出了1000量子比特的-量子计算机Condor Plus,这是全球历史上首台突破千量子比特大关的实用型量子计算机。IBM研究副总裁Jay Gambetta详细介绍了这套系统如何将量子-错误率降低了50%,使其能够稳定运行复杂的量子算法。这个项目汇聚了MIT、哈佛大学和众多知名制药公司的力量。该系统已经开始在药物研发和材料-科学研究中发挥作用,这一突破无疑将量子计算离商业化应用的距离又拉近了好几步。--春天的到来似乎预示着人类要探索更多的新边界。3月5日,Meta在加州门洛帕克总部举行了一场颠覆性的产品发布,推出了消费级脑机接口设备-BrainLink。Mark Zuckerberg在发布会上亲自演示了如何通过思维意念来控制虚拟现实环境中的对象,这场景看起来就像是从科幻大片里走出来的。-该设备采用了非侵入式的前沿设计,佩戴简便舒适,价格定为999美元。Meta与包括Harvard Medical School在内的多家医疗机构深度合作进行了临床试验,-参与者超过5000人。应用场景涵盖游戏、医疗康复和为残障人士提供生活辅助,业界普遍认为这将开启人机交互的全新时代。--把目光投向太空的同一个月,中国在3月20日于海南文昌航天发射场成功发射了"天宫商业1号"空间站核心舱。这是全球首个完全由私营企业主导的-商业空间站项目,打破了长期以来太空领域被国家宇航局垄断的局面。参与企业包括航天科技集团、华为和腾讯这样的科技巨头。该空间站设计容纳6名-宇航员,主要用于微重力科学实验和高端太空旅游体验。首批商业宇航员预计在6月份进驻。这个项目总投资达到了200亿元人民币,标志着中国商业航天-真正迈入了一个全新的发展阶段。--进入四月,硅谷的算力竞争开始白热化。谷歌在4月10日于加州山景城推出了第六代张量处理单元TPU v6,这款芯片采用了前沿的3纳米工艺。与上一代-相比,AI计算性能提升了5倍,能源效率提升了3倍。Sundar Pichai雄心勃勃地宣布计划在全球数据中心部署超过100万颗TPU v6芯片。谷歌云的客户-将获得优先使用权。这个举措给英伟达、AMD等竞争对手施加了巨大的压力,市场分析人士纷纷预测,整个AI芯片行业的竞争格局可能将被重新洗牌。--同月下旬的4月25日,生物医疗领域传来了令人欣喜的突破。美国FDA正式批准了首个基于CRISPR基因编辑技术的镰状细胞病疗法上市。临床试验在-包括波士顿儿童医院在内的10家权威医疗中心进行,参与的45名患者在接受治疗后全部康复。这个研究团队由诺贝尔奖得主Jennifer Doudna亲自领导。-这是CRISPR技术首次在遗传性疾病治疗中取得如此完美的成功。虽然每人的治疗费用预计为100万美元,但这个突破意义重大,为其他遗传性疾病的治疗-开辟了崭新的光明前景。--五月份的科技新闻继续让人应接不暇。亚马逊在5月8日于西雅图宣布了其送货无人机Fleet X的正式商用。Jeff Bezos在发布会上表示,新一代无人机-可以承载5公斤的货物,飞行距离达到30公里,超越了之前的所有技术指标。首批服务城市包括西雅图、洛杉矶和纽约这样的大都市,目前已经有超过-1000架无人机完成部署。通过这套系统,配送时间可以缩短至30分钟以内,这无疑将彻底改变最后一公里的物流体验。联邦航空局甚至为此制定了专门的-空域管理规则。这标志着无人机配送终于进入了规模化应用阶段。--月底的5月30日,能源领域传来了同样振奋的消息。美国劳伦斯利弗莫尔国家实验室宣布在核聚变实验中实现了3倍的能量增益,这是继去年首次实现-净能量增益之后的又一次重大突破。这个由200多名顶级科学家组成的研究团队利用世界上最强大的激光系统NIF取得了这一成就。能源部长亲自出来为这个-突破站台,表示这使商业化核聚变发电的目标更加近在咫尺。多家能源巨头已经投入数十亿美元来研发商用核聚变反应堆,而专家们普遍预测,一场清洁-能源革命可能在十年内就会到来,这将彻底改变全球能源格局。--"""---def main():- print("=" * 80)- print("AutoList 多 Chunk 提取示例")- print("=" * 80)-- # 1. 生成长文本- print("\n[1] 生成测试新闻文本...")- news_text = generate_long_news_text()- print(f" 文本长度: {len(news_text)} 字符")- print(f" 预计分块数: {len(news_text) // 500 + 1} 块 (chunk_size=500)")-- # 2. 初始化 LLM 和 Embedder- print("\n[2] 初始化 LLM 和 Embedder...")- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings(model="text-embedding-3-small")-- # 3. 创建 AutoList 实例 (使用小的 chunk_size 测试多 chunk)- print("\n[3] 创建 AutoList 实例 (chunk_size=500)...")- a_event_list = AutoList(- item_schema=NewsEvent,- llm_client=llm,- embedder=embedder,- chunk_size=500, # 设置小的 chunk_size 来测试多 chunk 提取- chunk_overlap=50,- max_workers=5,- )-- # 4. 提取知识 (使用 feed_text 模式)- print("\n[4] 提取新闻事件列表 (使用 feed_text 模式)...")- print("-" * 80)- a_event_list.feed_text(news_text)- print("-" * 80)-- # 5. 显示提取结果- print(f"\n[5] 提取到 {len(a_event_list.items)} 个新闻事件:")- for i, event in enumerate(a_event_list.items[:5], 1): # 只显示前5个- print(f"\n 事件 {i}:")- print(f" 标题: {event.title}")- print(f" 日期: {event.date}")- print(f" 地点: {event.location}")- print(f" 描述: {event.description[:80]}...")- print(f" 参与者: {', '.join(event.participants[:3])}")- if event.impact:- print(f" 影响: {event.impact[:60]}...")-- if len(a_event_list) > 5:- print(f"\n ... 还有 {len(a_event_list) - 5} 个事件未显示")-- # 6. 测试列表大小- print(f"\n[6] 知识列表大小: {len(a_event_list)} 个事件")-- # 7. 构建索引- print("\n[7] 构建向量索引...")- a_event_list.build_index()-- # 8. 测试搜索- print("\n[8] 测试事件搜索...")- queries = [- "人工智能相关的事件",- "美国发生的科技事件",- "与航天相关的新闻",- ]-- for query in queries:- print(f"\n 查询: '{query}'")- results = a_event_list.search(query, top_k=3)- for i, event in enumerate(results, 1):- print(f" [{i}] {event.title} ({event.date})")- print(f" 地点: {event.location}")-- # 9. 测试增量提取(feed 模式)- print("\n[9] 测试增量提取(feed 模式)...")- additional_news = """- 六月中旬的时候,通信领域又有了一个重磅消息。在深圳华为总部的一场技术发布大会上,华为的高管们宣布公司已经建成了全球首个6G技术- 测试网络,这消息一出来整个业界都沸腾了。这套测试网络覆盖了深圳市区的多个区域,实测传输速率达到了令人瞪目结舌的每秒1TB。这个项目- 汇聚了华为、中国移动和来自北大、清华等多家顶尖大学的研究力量。业界的普遍预测是,基于这样的技术进展,6G有望在2028年左右开始商用,- 这意味着全球通信技术的下一代竞争已经提前打响了。华为在这一轮竞争中处于相当领先的地位,这对中国的技术产业来说无疑是个重大利好消息。- """-- print(" 添加补充新闻...")- before_count = len(a_event_list)- a_event_list.feed_text(additional_news) # 使用 feed_text 自动合并- after_count = len(a_event_list)- print(f" 合并前: {before_count} 个事件")- print(f" 合并后: {after_count} 个事件")- print(f" 新增: {after_count - before_count} 个事件")-- # 10. 显示最新添加的事件- if after_count > before_count:- print("\n 最新添加的事件:")- for event in a_event_list.items[-3:]:- print(f" - {event.title} ({event.date})")-- a_event_list.build_index()- # 11. 保存知识- print("\n[10] 保存知识到文件...")- save_path = Path(__file__).parent.parent / "temp" / "auto_list_demo"- a_event_list.dump(str(save_path))- print(f" 已保存到: {save_path}")-- # 12. 重新加载测试- print("\n[11] 测试知识加载...")- a_loaded_event_list = AutoList(- item_schema=NewsEvent,- llm_client=llm,- embedder=embedder,- )- a_loaded_event_list.load(str(save_path))- print(f" 加载成功! 事件数: {len(a_loaded_event_list)}")- print(f" 第一个事件: {a_loaded_event_list.items[0].title}")-- # 13. 测试清空功能- print("\n[12] 测试清空功能...")- print(f" 清空前: {len(a_loaded_event_list)} 个事件")- a_loaded_event_list.clear()- print(f" 清空后: {len(a_loaded_event_list)} 个事件")-- # 14. 智能对话 (Chat)- print("\n" + "=" * 80)- print("💬 智能对话功能")- print("=" * 80)- print("正在基于提取的事件列表进行智能问答...\n")-- chat_questions = [- "这些新闻中,哪些事件对科技行业的影响最大?",- "请总结一下2024年科技领域的主要趋势。",- ]-- for q in chat_questions:- print(f"❓ 问题: {q}")- try:- resp = a_event_list.chat(q, top_k=5)- print(f"✨ 回答: {resp.content}\n")- except Exception as e:- print(f"⚠️ 对话出错: {e}\n")-- print("\n" + "=" * 80)- print("示例运行完成!")- print("=" * 80)-- a_event_list.show(- item_label_extractor=lambda item: item.title,- top_k_for_search=3,- top_k_for_chat=3,- )---if __name__ == "__main__":- main()diff --git a/examples/types/model_demo.py b/examples/types/model_demo.pydeleted file mode 100644index 728544b..0000000--- a/examples/types/model_demo.py+++ /dev/null@@ -1,265 +0,0 @@-"""-AutoModel 多 Chunk 提取示例--演示如何使用 AutoModel 处理长文本,测试多 chunk 并发提取和合并功能。--Usage:- python examples/types/model_demo.py-"""--import sys-from pathlib import Path--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from typing import List, Optional-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-from hyperextract import AutoModel--import dotenv--dotenv.load_dotenv()---# 定义文档知识 Schema-class DocumentSchema(BaseModel):- """文档综合知识结构"""-- title: str = Field(default="", description="文档标题")- author: str = Field(default="", description="作者姓名")- publish_date: str = Field(default="", description="发布日期")- category: str = Field(default="", description="文档类别")- summary: str = Field(default="", description="文档摘要")- main_topics: List[str] = Field(default_factory=list, description="主要话题")- key_findings: List[str] = Field(default_factory=list, description="关键发现")- methodology: str = Field(default="", description="研究方法")- conclusion: str = Field(default="", description="结论")- keywords: List[str] = Field(default_factory=list, description="关键词")---def generate_long_document() -> str:- """生成一个长文档用于测试多 chunk 提取"""- return """-# 人工智能在医疗诊断中的应用研究报告--作者: 张伟博士-发布日期: 2024年3月15日-类别: 医疗科技研究--## 摘要--本研究报告深入探讨了人工智能技术在现代医疗诊断领域的广泛应用。通过分析大量临床数据和案例研究,-我们发现AI技术在提高诊断准确率、降低医疗成本、优化治疗方案等方面展现出巨大潜力。--## 第一部分:引言与背景--人工智能(AI)技术的快速发展正在深刻改变医疗行业的面貌。特别是在医疗诊断领域,AI展示了超越传统-方法的潜力。深度学习、计算机视觉和自然语言处理等技术的结合,使得AI系统能够从海量医疗数据中-学习并识别疾病模式。--随着电子健康记录(EHR)的普及和医学影像数字化,医疗数据呈现爆炸式增长。这为AI技术的应用提供了-丰富的训练素材。同时,计算能力的提升和算法的优化,使得复杂的医学诊断任务变得更加可行。--## 第二部分:研究方法与数据收集--本研究采用了多种研究方法相结合的方式:--1. **文献综述**: 系统性回顾了2015-2024年间发表的相关学术论文超过500篇-2. **案例分析**: 深入分析了15家医院实施AI诊断系统的实际案例-3. **实验研究**: 在合作医院进行了为期18个月的临床试验-4. **专家访谈**: 采访了50位医疗专家和AI研究人员--数据收集方面,我们使用了包括X光片、CT扫描、MRI影像在内的超过100,000份医学影像数据,-以及相应的诊断报告和患者病历。所有数据均经过严格的隐私保护处理,符合医疗数据使用规范。--## 第三部分:主要发现--### 3.1 诊断准确率显著提升--研究发现,AI辅助诊断系统在多个疾病领域的准确率达到了专家级水平:-- 肺癌早期检测准确率达到94.5%,比传统方法提高12%-- 糖尿病视网膜病变识别准确率达到96.8%-- 皮肤癌分类准确率达到91.2%--### 3.2 诊断效率大幅提高--AI系统能够在数秒内完成原本需要医生数小时才能完成的影像分析工作。平均诊断时间从45分钟-缩短至3分钟,效率提升了15倍。--### 3.3 成本效益分析--实施AI诊断系统后,医院的诊断成本平均降低了35%。主要节省来自于:-- 减少了不必要的复诊-- 降低了误诊率-- 优化了医疗资源配置--### 3.4 早期疾病检测--AI系统在早期疾病检测方面表现出色,能够识别出人眼难以察觉的微小病变。这对于提高治疗成功率-和患者生存率具有重要意义。--## 第四部分:技术实现与挑战--### 4.1 核心技术--研究涉及的主要AI技术包括:-- 卷积神经网络(CNN)用于医学影像分析-- 循环神经网络(RNN)用于处理时序医疗数据-- 迁移学习技术加速模型训练-- 联邦学习保护患者隐私--### 4.2 面临的挑战--尽管AI在医疗诊断中展现出巨大潜力,但仍面临诸多挑战:-1. 数据质量和标注问题-2. 模型可解释性不足-3. 医疗法规和伦理问题-4. 医生接受度和信任问题-5. 系统集成和部署成本--## 第五部分:结论与展望--通过本研究,我们得出以下结论:--1. AI技术已经在医疗诊断领域取得了显著成果,能够有效提高诊断准确率和效率-2. AI不是要替代医生,而是作为强有力的辅助工具,帮助医生做出更准确的诊断-3. 数据质量、算法透明度和医疗伦理是未来需要重点关注的问题-4. 跨学科合作(医学专家+AI研究人员)是推动该领域发展的关键--展望未来,我们预期AI在医疗诊断领域将继续深化应用,特别是在个性化医疗、远程诊断、-罕见病识别等方面将发挥更大作用。同时,随着可解释AI和联邦学习等技术的成熟,-AI医疗诊断系统的可信度和安全性将进一步提升。--## 关键词--人工智能、医疗诊断、深度学习、医学影像分析、诊断准确率、医疗AI、计算机辅助诊断、-临床应用、机器学习、医疗科技-"""---def main():- print("=" * 80)- print("AutoModel 多 Chunk 提取示例")- print("=" * 80)-- # 1. 生成长文档- print("\n[1] 生成测试文档...")- document = generate_long_document()- print(f" 文档长度: {len(document)} 字符")- print(f" 预计分块数: {len(document) // 500 + 1} 块 (chunk_size=500)")-- # 2. 初始化 LLM 和 Embedder- print("\n[2] 初始化 LLM 和 Embedder...")- llm_client = ChatOpenAI(model="gpt-4o-mini")- embedder = OpenAIEmbeddings(model="text-embedding-3-small")-- # 3. 创建 AutoModel 实例 (使用小的 chunk_size 测试多 chunk)- print("\n[3] 创建 AutoModel 实例 (chunk_size=500)...")- a_document_model = AutoModel(- data_schema=DocumentSchema,- llm_client=llm_client,- embedder=embedder,- chunk_size=500, # 设置小的 chunk_size 来测试多 chunk 提取- chunk_overlap=50,- max_workers=5,- )-- # 4. 提取知识 (使用 feed_text 模式)- print("\n[4] 提取知识 (使用 feed_text 模式)...")- print("-" * 80)- a_document_model.feed_text(document)- extracted_data = a_document_model.data- print("-" * 80)-- # 5. 显示提取结果- print("\n[5] 提取结果:")- print(f" 标题: {extracted_data.title}")- print(f" 作者: {extracted_data.author}")- print(f" 发布日期: {extracted_data.publish_date}")- print(f" 类别: {extracted_data.category}")- print(f" 摘要: {extracted_data.summary[:100]}...")- print(f" 主要话题: {extracted_data.main_topics}")- print(f" 关键发现数量: {len(extracted_data.key_findings)}")- print(f" 方法论: {extracted_data.methodology[:100]}...")- print(f" 结论: {extracted_data.conclusion[:100]}...")- print(f" 关键词数量: {len(extracted_data.keywords)}")-- # 6. 构建索引- print("\n[6] 构建向量索引...")- a_document_model.build_index()-- # 7. 测试搜索- print("\n[7] 测试知识搜索...")- queries = [- "研究的主要发现是什么",- "使用了哪些研究方法",- "AI的准确率如何",- ]-- for query in queries:- print(f"\n 查询: '{query}'")- results = a_document_model.search(query, top_k=2)- for i, result in enumerate(results, 1):- field_name = list(result.keys())[0]- field_value = result[field_name]- print(f" [{i}] {field_name}: {str(field_value)[:80]}...")-- # 8. 测试增量提取(feed_text 模式)- print("\n[8] 测试增量提取(feed_text 模式)...")- additional_text = """- 补充信息:- 本研究获得了国家自然科学基金支持,项目编号:NSFC-2023-12345。- 研究团队来自清华大学医学院和北京协和医院。- """- print(" 添加补充文本...")- a_document_model.feed_text(additional_text) # 使用 feed_text 自动合并- print(f" 更新后的作者信息: {a_document_model.data.author}")-- # 9. 保存知识- print("\n[9] 保存知识到文件...")- save_path = Path(__file__).parent.parent / "temp" / "auto_model_demo"- a_document_model.dump(str(save_path))- print(f" 已保存到: {save_path}")-- # 10. 重新加载测试- print("\n[10] 测试知识加载...")- a_loaded_document_model = AutoModel(- data_schema=DocumentSchema,- llm_client=llm_client,- embedder=embedder,- )- a_loaded_document_model.load(str(save_path))- print(" 加载成功!")- print(f" 标题: {a_loaded_document_model.data.title}")-- # 11. 智能对话 (Chat)- print("\n" + "=" * 80)- print("💬 文档智能咨询")- print("=" * 80)- print("基于提取的文档结构进行智能对话...\n")-- doc_questions = [- "这篇文档的核心观点是什么?",- "文档中提到的主要技术有哪些?"- ]-- a_document_model.build_index()- for q in doc_questions:- print(f"❓ 读者: {q}")- try:- resp = a_document_model.chat(q, top_k=3)- print(f"📖 AI助手: {resp.content}\n")- except Exception as e:- print(f"⚠️ 回答失败: {e}\n")-- a_document_model.show(label_extractor=lambda x: x.title)-- print("\n" + "=" * 80)- print("示例运行完成!")- print("=" * 80)---if __name__ == "__main__":- main()diff --git a/examples/types/set_demo.py b/examples/types/set_demo.pydeleted file mode 100644index 80ac704..0000000--- a/examples/types/set_demo.py+++ /dev/null@@ -1,289 +0,0 @@-"""-集合知识示例 - 从冒险者日志中提取和去重怪物信息--这个示例演示了AutoSet如何基于唯一键字段自动去重提取的项目。-使用RPG怪物图鉴场景展示LLM_MERGE策略如何智能合并来自多个来源的冲突信息。--Usage:- python examples/types/set_demo.py-"""--import sys-from pathlib import Path--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-from ontomem.merger import MergeStrategy-from hyperextract import AutoSet--import dotenv--dotenv.load_dotenv()---# 定义怪物信息的数据模型-class MonsterEntry(BaseModel):- """RPG游戏中的怪物信息"""-- name: str = Field(description="怪物的名称(例如:'地精小卒'、'洞穴巨魔')")- species: str | None = Field(- None,- description="怪物的种族分类(例如:'类人型'、'巨人'、'野兽')",- )- danger_level: int | None = Field(- None,- description="危险等级(1-10),用于评估战斗难度",- )- habitats: str | None = Field(None, description="怪物常见的栖息地或活动场所")- characteristics: str | None = Field(- None, description="怪物的外观特征和行为习性描述"- )- weaknesses: str | None = Field(- None, description="怪物对特定伤害类型或攻击方式的弱点"- )- loot_drops: str | None = Field(None, description="击败怪物可能掉落的物品和资源")---def main():- # 初始化LLM和嵌入模型- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings(model="text-embedding-3-small")-- # 创建AutoSet,使用LLM_MERGE策略- # 当同一个怪物在多个日志中出现时,LLM会智能合并冲突的信息- monster_codex = AutoSet(- item_schema=MonsterEntry,- llm_client=llm,- embedder=embedder,- key_extractor=lambda x: x.name, # 基于怪物名称去重- strategy_or_merger=MergeStrategy.LLM.PREFER_INCOMING,- prompt="""从冒险者的日志记录中提取所有怪物的相关信息。--对于每只怪物,请提供:-- name: 怪物的确切名称-- species: 怪物的种族分类(类人型、野兽、不死生物等)-- danger_level: 战斗危险等级(1-10)-- habitats: 常见的栖息地或地理位置-- characteristics: 详细的外观特征和行为描述-- weaknesses: 已知的弱点和克制方法-- loot_drops: 击败后可能掉落的物品--要尽可能准确和详细,包括所有提到的怪物和它们的属性。如果信息不完整,请基于已有信息做出合理推断。--### 冒险者日志:-{source_text}""",- )-- # 来自冒险者的日志 - 包含重复的怪物但信息略有不同- adventurer_logs = [- """- 【第一个冒险日记 - 林地远征】- 日期:秋月15日-- 今天在幽暗森林中遭遇了几种怪物。首先遇到了一群地精小卒,他们大约有3-4英尺高,绿皮肤,- 穿着破烂的皮盔甲。这些家伙相当狡猾,喜欢群体狩猎。危险等级大概是2-3。它们掉落了- 一些铜币和破旧的匕首。-- 在森林深处,我还遇到了一只洞穴巨魔。这个家伙大约8英尺高,皮肤呈灰褐色,力量惊人。- 对火焰特别敏感——我用火焰咒语成功地逼退了它。危险等级至少是7。它的巢穴里有一些金币。-- 还有森林狼群,非常聪明的野兽,它们似乎有高度的群体智能。危险等级4-5。- """,- """- 【第二个冒险日记 - 深矿探险】- 日期:秋月18日-- 在深矿洞中进行了危险的探险。地精小卒在这里也有出现!看来他们在地下还有殖民地。- 这次的地精穿着更好的装备,显然这是地精的精英部队。他们配备了铁制短剑,训练更充分。- 危险等级可能达到了3-4。掉落了更多的铜币和一些魔晶。-- 还有一种我之前没遇过的怪物——地下爬行虫。这些生物有分节的身体,长约2-3米,- 用触须来感知周围环境。它们不是很聪明,但速度很快。危险等级3。掉落虫壳碎片。-- 巨魔在深处也出现了,但这次的巨魔看起来比森林里的那只更大更凶悍。- 它对寒冰魔法有抵抗力,但仍然惧怕火焰。危险等级估计8级。- """,- """- 【第三个冒险日记 - 废墟遗迹】- 日期:秋月22日-- 在古老遗迹的探索中发现了新的敌人类型。最常见的是地精小卒的变种——遗迹守卫。- 他们看起来是被之前的文明改造过的,具有更强的战斗能力。有人说他们被魔法增强过。- 危险等级跳到了4-5。这些怪物掉落了古代金币和遗迹碎片。-- 还有一个更大的怪物——遗迹看守者,是一只3米高的怪物。它有三只眼睛,可能具有某种魔法能力。- 非常危险!危险等级至少9级。击败它时获得了一个古老的魔晶。-- 巨魔在这里也有据点,防守非常严密。这些可能是智商更高的个体。- 它们使用简单的工具和陷阱来狩猎冒险者。危险等级8-9。- """,- ]-- print("=" * 80)- print("集合知识演示:从冒险者日志中构建怪物图鉴")- print("=" * 80)-- # 从所有日志中提取怪物信息- print("\n[*] 正在处理冒险者日志...")- for i, log in enumerate(adventurer_logs, 1):- print(f"\n 处理日志 {i}...")- monster_codex.feed_text(log) # 喂养数据- print(f" √ 目前已识别的怪物数量:{len(monster_codex)}")-- # 显示所有提取的怪物- print("\n" + "=" * 80)- print(f"[*] 提取的怪物统计:共 {len(monster_codex)} 种不同的怪物")- print("=" * 80)-- # 按危险等级分组显示- monsters_by_danger = {}- for monster in monster_codex.items:- level = monster.danger_level or 0- if level not in monsters_by_danger:- monsters_by_danger[level] = []- monsters_by_danger[level].append(monster)-- # 从低到高显示怪物- print("\n[怪物按危险等级分类]\n")- for level in sorted(monsters_by_danger.keys()):- level_monsters = monsters_by_danger[level]- print(f"[!] 危险等级 {level}:")- for monster in sorted(level_monsters, key=lambda m: m.name):- print(f"\n ● 怪物名称:{monster.name}")- if monster.species:- print(f" 种族:{monster.species}")- if monster.habitats:- print(f" 栖息地:{monster.habitats}")- if monster.characteristics:- print(f" 特征:{monster.characteristics}")- if monster.weaknesses:- print(f" 弱点:{monster.weaknesses}")- if monster.loot_drops:- print(f" 掉落物:{monster.loot_drops}")-- # 演示集合操作- print("\n" + "=" * 80)- print("[*] 演示集合操作和语义搜索")- print("=" * 80)-- # 构建向量索引以支持语义搜索- print("\n正在构建向量索引...")- monster_codex.build_index()- print("√ 索引构建成功")-- # 语义搜索示例- print("\n[1] 语义搜索 - 寻找掉落金币的怪物:")- treasure_monsters = monster_codex.search("掉落金币财宝", top_k=5)- for monster in treasure_monsters:- print(f" • {monster.name}")- if monster.loot_drops:- print(f" 掉落物:{monster.loot_drops}")-- print("\n[2] 语义搜索 - 寻找怕火焰的怪物:")- fire_weak_monsters = monster_codex.search("害怕火焰弱点火", top_k=5)- for monster in fire_weak_monsters:- print(f" • {monster.name}")- if monster.weaknesses:- print(f" 弱点:{monster.weaknesses}")-- print("\n[3] 语义搜索 - 寻找在地下出现的怪物:")- underground_monsters = monster_codex.search("地下洞穴矿井", top_k=5)- for monster in underground_monsters:- print(f" • {monster.name}")- if monster.habitats:- print(f" 栖息地:{monster.habitats}")-- # 检查特定怪物是否存在- print("\n[4] 成员检验 - 检查特定怪物是否在图鉴中:")- check_monsters = ["地精小卒", "洞穴巨魔", "龙", "骷髅战士"]- for monster_name in check_monsters:- exists = monster_codex.contains(monster_name)- status = "√" if exists else "×"- print(f" {status} {monster_name}:{'已记录' if exists else '未记录'}")-- # 获取特定怪物的详细信息- print("\n[5] 查询特定怪物详情:")- goblin = monster_codex.get("地精小卒")- if goblin:- print(f" 怪物名称:{goblin.name}")- print(f" 种族:{goblin.species or '未知'}")- print(f" 危险等级:{goblin.danger_level or '未知'}/10")- print(f" 栖息地:{goblin.habitats or '未知'}")- print(f" 特征:{goblin.characteristics or '无'}")- print(f" 弱点:{goblin.weaknesses or '无已知弱点'}")- print(f" 掉落物:{goblin.loot_drops or '无'}")-- # 保存提取的知识- print("\n" + "=" * 80)- print("[*] 保存怪物图鉴")- print("=" * 80)-- save_path = project_root / "temp" / "auto_set_demo"- monster_codex.dump(save_path)- print(f"√ 已保存到:{save_path}")- print(" - state.json(结构化数据)")- print(" - faiss_index/(向量索引)")-- # 演示加载功能- print("\n[*] 加载已保存的怪物图鉴...")- loaded_codex = AutoSet(- item_schema=MonsterEntry,- llm_client=llm,- embedder=embedder,- key_extractor=lambda x: x.name,- )- loaded_codex.load(save_path)- print(f"√ 加载成功!图鉴中有 {len(loaded_codex)} 只怪物")-- # 显示已加载的怪物列表- print("\n[已加载怪物列表]")- for monster in loaded_codex.items:- print(f" • {monster.name} (危险等级:{monster.danger_level}/10)")-- # 总结- print("\n" + "=" * 80)- print("[*] 演示总结")- print("=" * 80)- print(f" 提取的不同怪物数量:{len(monster_codex)}")- print(f" 处理的冒险者日志数:{len(adventurer_logs)}")- print(" 去重策略:LLM_MERGE(智能合并)")- print(" 去重键:lambda x: x.name")- print("\n √ 基于怪物名称自动去重")- print(" √ 智能合并多个来源的冲突信息")- print(" √ 支持所有怪物的语义搜索")- print(" √ 持久化存储和加载")-- # 交互式对话 (Chat)- print("\n" + "=" * 80)- print("💬 冒险者智能咨询系统")- print("=" * 80)- print("根据怪物图鉴回答冒险者的提问...\n")-- adventurer_questions = [- "我想去地下矿井冒险,应该带什么装备来对付那里的怪物?",- "哪些怪物掉落最值钱的宝物?",- "请给我推荐一个适合新手的狩猎对象。"- ]-- for q in adventurer_questions:- print(f"⚔️ 冒险者: {q}")- try:- answer = monster_codex.chat(q, top_k=5)- print(f"🧙 NPC商人: {answer.content}\n")- except Exception as e:- print(f"⚠️ 无法回答: {e}\n")-- print("=" * 80)-- monster_codex.show(- item_label_extractor=lambda x: x.name,- top_k_for_search=3,- top_k_for_chat=3- )---if __name__ == "__main__":- main()diff --git a/examples/types/spatial_graph_demo.py b/examples/types/spatial_graph_demo.pydeleted file mode 100644index c33c5da..0000000--- a/examples/types/spatial_graph_demo.py+++ /dev/null@@ -1,349 +0,0 @@-"""-AutoSpatialGraph Demo: 深海惊魂 - "阿特拉斯"号的幽灵 🌊👻--场景背景:-一名深海打捞专家进入失联的科研站"阿特拉斯"号。由于电力系统故障,导航地图失效。-主角必须通过观察周围环境、阅读墙壁标识和逻辑推理来确定自己在空间站的具体位置。--难点:-1. 3D 空间结构:不仅有东南西北,还有"上层甲板"、"下层动力舱"、"垂直维修井"。-2. 相对位置推理:"这里的阀门控制着楼下的..."、"头顶传来了..."。-3. 空间作为属性:位置不再是节点,而是实体(尸体、设备、怪物)的固有属性。-4. 观察点注入:所有相对描述("这里")必须解析为主角当前的观察点。--Usage:- python examples/types/spatial_graph_demo.py-"""--import sys-from pathlib import Path-from dotenv import load_dotenv--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from typing import Optional-from langchain_openai import ChatOpenAI, OpenAIEmbeddings--from hyperextract.types import AutoSpatialGraph-from ontomem import MergeStrategy--load_dotenv()--# ==================== 1. 定义空间感知知识结构 (Schema) ====================---class DeepSeaEntity(BaseModel):- """深海空间站实体节点"""-- name: str = Field(description="实体名称(人名、设备名、怪物代号、房间名)")- category: str = Field(- description="实体类型:'船员', '怪物', '关键设备', '房间/区域', '物品'",- default="未知",- )- status: str = Field(- description="当前状态(例如:已死亡、损坏、锁定、活跃)", default="正常"- )- description: str = Field(description="实体的详细描述")-- def __repr__(self):- return f"🕵️ {self.name} ({self.status}) [{self.category}]"---class InteractionEdge(BaseModel):- """实体间的互动关系 (包含空间信息)"""-- source: str = Field(description="发起者")- target: str = Field(description="对象")- relation: str = Field(description="关系或动作(例如:'攻击','修复','操作','持有/携带')")- location: Optional[str] = Field(description="互动发生的具体位置", default=None)- details: str = Field(description="互动的详细描述")-- def __repr__(self):- loc_str = f" @ {self.location}" if self.location else ""- return f"⚡ {self.source} --[{self.relation}]--> {self.target}{loc_str}"---# ==================== 2. 深海探险日志文本 ====================--DEEP_SEA_LOG = """-【自动日志转录 - 救援队员编号:R-09】-【任务:阿特拉斯(Atlas)轨道工厂 - 重启冷却系统】-【当前环境:重力异常,高温警报】-【初始观察点:主气闸舱 (Airlock Alpha)】--08:00-气闸舱的警示灯在闪烁。我刚对接进入「主气闸舱」。这里的空气中弥漫着烧焦的臭氧味。-在我面前(气闸舱正中央)是两台处于休眠状态的安保机器人,型号分别是 "Sentinel-01" 和 "Loader-Bot Beta"。-Sentinel-01 的机械臂严重扭曲,似乎是被某种巨大的力量强行折断的,而 Loader-Bot Beta 的抓手里紧紧握着一张蓝色的 "工程维护卡"。--08:15-维护卡显示拥有「高能物理实验室-B区」的权限。-我看向气闸舱的左侧,那里的重型隔离门上喷着 "To Crew Quarters (船员休眠区)" 的字样,门禁面板显示"封锁中"。-右侧的舱门半开着,门楣上写着 "To Central Hub (中央枢纽)"。地上有一道明显的机油泄露痕迹,延伸进右侧的黑暗走廊中。--08:30-我穿过右侧舱门,进入了「中央枢纽」。这是一个巨大的圆形大厅,占据了空间站的主层。-这里的上方是网格状的金属地板。透过网格,我能看到头顶上方(上层甲板)悬挂着巨大的「备用冷却罐」。那个冷却罐表面覆盖着厚厚的冰霜,显然发生了泄露。-大厅的正中央是一座全息控制台,现在正不断弹出红色的错误代码。--08:45-顺着中央枢纽的地板格栅往下看,我能隐约看到下方的「动力反应堆舱」。那里闪烁着危险的橙色光芒。-突然,反应堆舱那个方向传来了一声金属爆裂的巨响。有什么设备炸了。-我想去找通往下层的梯子,但在枢纽的北侧墙壁上,我发现了一张破损的结构图。如果图纸没更新,那里正是「核心控制阀」所在的位置。--09:00-我决定先去寻找备用电源。根据记忆,就在这个大厅(中央枢纽)的东侧回廊尽头。-我走进东侧回廊。路过「维修车间」时,自动门突然卡住了。-通过门缝,我看到维修车间的一张工作台上,放着被称为 "Prototype-X" 的实验型外骨骼。它看起来像是被拆解了……外壳被暴力撕开。-最关键的是,在这台外骨骼的胸腔核心位置,插着一枚闪烁的「系统引导盘」。--09:10-我试图撬开门取出引导盘。-这时,工作台旁边的阴影里窜出了一只小型的 "Scavenger-Drone (清道夫无人机)"。它处于失控暴走状态。-它向我发射了激光切割束。我举起电磁脉冲盾,将它弹飞到旁边的工具柜上。这里的工具柜被撞得粉碎。--09:20-拿到系统引导盘后,我必须去恢复电力才能重启冷却。-回到中央枢纽,我找到了通往顶部的维修梯。我爬上了头顶的「上层甲板」。-这里的重力系统失效了,冷却液凝结成的冰晶在空中漂浮。-在上层甲板的最南端,就是「主控室」。透过破碎的观察窗,我看到里面的情况很糟。-站长AI终端 "Master Computer" 的屏幕完全碎裂。-终端机的数据接口上插着一根不明来源的数据线,一直连接到地板下的线路中。--09:30-我走进主控室。Master Computer 发出了断断续续的合成音,声音从四周的扬声器(位于主控室墙壁)里传出来:-"警告……核心温度临界……无法控制……"-我看了一眼备用屏幕。监控显示,刚才发出爆炸声的「核心控制阀」(位于我们正下方的底层/反应堆舱)已经被熔化的金属覆盖。-而那种高温熔融物,正通过通风管道,从底层慢慢向上层的各个房间蔓延。--09:45-我必须做出选择。是立刻撤离,还是尝试手动关闭反应堆?-无论如何,我现在的位置(主控室)温度正在急剧升高。通风管道里吹出了灼热的风。--09:50-我冲出主控室,回到上层甲板。这时我发现了一个关键信息——在备用冷却罐下方贴着一份维护日志:-"实验机体 Prototype-X 的核心电池极不稳定。原计划移送至 高能物理实验室-B区 进行拆解,暂存维修车间。"-这意味着,Prototype-X 原本是要送去「高能物理实验室-B区」的。而那间实验室的权限卡,正在主气闸舱里 Loader-Bot Beta 的手中。--10:00-我需要获取那张维护卡,但这意味着我要冒着高温熔穿地板的风险,返回主气闸舱。-在我犹豫的时候,我听到了休眠区的方向传来了结构断裂的声音。-通过内部通讯系统,我听到了一段自动广播:-"注意……高能物理实验室-B区……结构性……崩溃……电池组……连锁反应……"--10:15-我的决定已经定了。我必须到达「高能物理实验室-B区」切断电池组。-为了到达那里,我有两条路:-一是返回主气闸舱取维护卡(路径较远但温度较低)。-二是通过维修通道的紧急截断阀(位于底层的动力反应堆舱侧面)绕过门禁系统(不仅路远,而且是高温核心区,危险系数极大)。-"""--# ==================== 3. 运行提取 ====================---def run_demo():- print("=" * 70)- print("🌊 深海惊魂 - 空间感知分析模块启动")- print("=" * 70)-- # 初始化 LLM- print("\n🔧 初始化 LLM 和嵌入模型...")- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings()-- # 创建 AutoSpatialGraph- print("📊 构建 AutoSpatialGraph 实例...\n")- graph = AutoSpatialGraph[DeepSeaEntity, InteractionEdge](- node_schema=DeepSeaEntity,- edge_schema=InteractionEdge,- node_key_extractor=lambda x: x.name,- # 基础边 Key- edge_key_extractor=lambda x: f"{x.source}|{x.relation}|{x.target}",- # 空间提取器移至边上- location_in_edge_extractor=lambda x: x.location or "",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder,- observation_location="Atlas深海科研站",- extraction_mode="two_stage",- node_strategy_or_merger=MergeStrategy.LLM.BALANCED,- edge_strategy_or_merger=MergeStrategy.LLM.BALANCED,- prompt_for_node_extraction=(- "提取深海空间站中的所有关键实体(船员、设备、怪物等)。\n"- "注意:不要在节点中提取位置信息,位置将作为关系属性提取。\n\n"- "## 源文本:\n"- "{source_text}"- ),- prompt_for_edge_extraction=(- "提取实体之间的互动关系,并精确捕获它们发生的地理位置。\n"- "特别是:\n"- "- 解析相对位置(如'这里'解析为 observation_location)\n"- "- 注意 3D 空间层级(上层甲板、中央枢纽、反应堆舱)"- "### 已知的实体列表\n"- "{known_nodes}\n\n"- "## 源文本:\n"- "{source_text}\n\n"- ),- chunk_size=2048,- )-- print(f"\n📖 正在加载深海探险日志...")- print("=" * 70)- print(DEEP_SEA_LOG[:300] + "...")- print("=" * 70)-- # 执行提取- print("\n⚙️ 执行空间感知提取(解析相对方位词)...\n")- graph.feed_text(DEEP_SEA_LOG)-- # ==================== 4. 空间分析结果 ====================-- print("\n" + "=" * 70)- print(f"✅ 空间结构分析完成!")- print("=" * 80)- print(f" 提取的实体: {len(graph.nodes)} 个")- print(f" 记录的互动关系: {len(graph.edges)} 个")-- print("\n" + "-" * 70)- print("🕵️ 空间站实体清单 (Nodes)")- print("-" * 70)- for i, node in enumerate(graph.nodes, 1):- print(f" {i}. {node}")-- print("\n" + "-" * 70)- print("📍 空间互动时间表 (Edges - 按位置排序)")- print("-" * 70)-- # 按位置排序打印关系- sorted_edges = sorted(graph.edges, key=lambda x: x.location or "ZZZ")- for i, edge in enumerate(sorted_edges, 1):- print(f" {i}. {edge}")-- # ==================== 5. 空间情报检索 (Search 阶段) ====================-- print("\n" + "=" * 70)- print("🔍 空间站导航系统 - 向量检索阶段")- print("=" * 70)-- # 建立索引- print("\n🏗️ 构建向量索引...")- graph.build_index()-- # 场景1:找怪物- q1 = "空间站里有哪些怪物或威胁?它们在哪?"- print(f"\n❓ Search Query: {q1}")- try:- nodes, edges = graph.search(q1, top_k_nodes=5, top_k_edges=5)- if nodes:- print(f" 📍 相关实体:")- for n in nodes:- print(f" - {n}")- if edges:- print(f" 🔗 相关关系:")- for e in edges:- print(f" - {e}")- if not nodes and not edges:- print(" (无相关记录)")- except Exception as e:- print(f" 检索失败: {e}")-- # 场景2:找关键物品- q2 = "黑匣子在哪里找到的?"- print(f"\n❓ Search Query: {q2}")- try:- nodes, edges = graph.search(q2, top_k_nodes=3, top_k_edges=3)- if nodes:- print(f" 📍 相关实体:")- for n in nodes:- print(f" - {n}")- if edges:- print(f" 🔗 相关关系:")- for e in edges:- print(f" - {e}")- if not nodes and not edges:- print(" (无相关记录)")- except Exception as e:- print(f" 检索失败: {e}")-- # 场景3:3D 空间理解- q3 = "动力反应堆舱(底层)里有什么东西?"- print(f"\n❓ Search Query: {q3}")- try:- nodes, edges = graph.search(q3, top_k_nodes=5, top_k_edges=5)- if nodes:- print(f" 📍 相关实体:")- for n in nodes:- print(f" - {n}")- if edges:- print(f" 🔗 相关关系:")- for e in edges:- print(f" - {e}")- if not nodes and not edges:- print(" (无相关记录)")- except Exception as e:- print(f" 检索失败: {e}")-- # 场景4:权限卡位置- q4 = "权限卡在哪里?"- print(f"\n❓ Search Query: {q4}")- try:- nodes, edges = graph.search(q4, top_k_nodes=3, top_k_edges=3)- if nodes:- print(f" 📍 相关实体:")- for n in nodes:- print(f" - {n}")- if edges:- print(f" 🔗 相关关系:")- for e in edges:- print(f" - {e}")- if not nodes and not edges:- print(" (无相关记录)")- except Exception as e:- print(f" 检索失败: {e}")--- # ==================== 6. 交互式情报问答 (Chat 阶段) ====================-- print("\n" + "=" * 70)- print("💬 深海空间站 - 交互式情报分析终端")- print("=" * 70)- print("系统提示: 您现在可以直接用自然语言询问关于空间结构、实体关系和危险评估的复杂问题。\n")-- chat_queries = [- "这个空间站里发生了什么悲剧?请根据现场的尸体、伤口和位置信息推断事件的发展顺序。",- "Subject X 是如何逃出医务室的?它现在可能在空间站的哪些地方?",- "潜水员如果要从医务室安全抵达生物实验室-B区,应该如何规划路线?需要注意哪些危险?",- ]-- for i, q in enumerate(chat_queries, 1):- print(f"👤 [深海调查官 {i}]: {q}")- print("🤖 [AI 空间分析引擎]: 正在检索空间拓扑结构并生成情报分析报告...")-- try:- # 使用 chat 接口进行 RAG 问答- response = graph.chat(q)- print(f"📄 [情报简报]:\n{response.content}\n")- except Exception as e:- print(f"⚠️ 系统响应异常: {e}\n")-- # ==================== 7. 保存 ====================-- print("\n" + "=" * 70)- print("💾 保存空间知识图谱...")- print("=" * 70)- save_path = project_root / "temp" / "auto_spatial_graph_demo"- try:- graph.dump(str(save_path))- print(f" 已保存至: {save_path}")- except Exception as e:- print(f" 保存失败: {e}")-- print("\n" + "=" * 70)- print("✅ 演示完成!")- print("=" * 70)---if __name__ == "__main__":- run_demo()diff --git a/examples/types/spatio_temporal_graph_demo.py b/examples/types/spatio_temporal_graph_demo.pydeleted file mode 100644index 8ba1227..0000000--- a/examples/types/spatio_temporal_graph_demo.py+++ /dev/null@@ -1,187 +0,0 @@-"""-AutoSpatioTemporalGraph Demo: 谍战惊悚 - 柏林的48小时 (时空增强版) 🕵️♀️🏙️--场景背景:-在冷战时期的东柏林,特工 Elias 的行动不仅在时间线上推进,其地理位置的变化也至关重要。-本 Demo 演示如何同时提取和处理:-1. 时间信息 (Timestamp)-2. 空间信息 (Location/Coordinates)-3. 时空上下文注入 (Observation Time & Location)--核心特性:-- 分离的时间/空间提取器 (Time & Location Extractors)-- 自动融合的时空唯一键 (Composite Spatio-Temporal Key)-- 相对时空消歧 (Relative Spatio-Temporal Resolution)--Usage:- python examples/types/spatio_temporal_graph_demo.py-"""--import sys-from pathlib import Path-from typing import Optional--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from langchain_openai import ChatOpenAI, OpenAIEmbeddings--from hyperextract.types import AutoSpatioTemporalGraph--import dotenv--dotenv.load_dotenv()--# ==================== 1. 定义时空感知知识结构 (Schema) ====================--class SpyEntity(BaseModel):- """谍战实体节点"""- name: str = Field(description="实体名称")- category: str = Field(description="实体类型:'特工', '地点', '组织', '物品'", default="未知")- description: str = Field(description="实体的详细描述", default="")-- def __repr__(self):- return f"🕵️ [{self.category}] {self.name}"--class SpatioTemporalAction(BaseModel):- """时空感知的行动边"""- source: str = Field(description="行动发起者")- target: str = Field(description="行动对象")- action: str = Field(description="具体行动")- time: Optional[str] = Field(description="发生的具体时间 (YYYY-MM-DD HH:MM)", default=None)- location: Optional[str] = Field(description="发生的具体地点", default=None)- details: str = Field(description="行动细节描述", default="")-- def __repr__(self):- ctx = []- if self.time: ctx.append(f"⏰ {self.time}")- if self.location: ctx.append(f"📍 {self.location}")- ctx_str = f" [{' '.join(ctx)}]" if ctx else ""- return f"⚡ {self.source} --[{self.action}]--> {self.target}{ctx_str}"--# ==================== 2. 叙述文本 (包含丰富的时空信息) ====================--ST_NARRATIVE = """-【任务简报:代号"碎冰"】-【当前基准:2024-03-15,柏林勃兰登堡门】--08:30,Elias 抵达了勃兰登堡门。他在广场西侧的长椅上坐下,距离标志性建筑仅50米。他正在等待接头指令。-五分钟后(08:35),Sarah 骑车从他身边经过,故意丢下一份夹着代码的《柏林日报》。--09:00,Elias 穿过街道,进入了位于 Friedrichstraße 车站附近的一个隐藏安全屋。他在地下室的一层准备了电子干扰器。-与此同时(09:00),主要目标 Viktor 出现在亚历山大广场(Alexanderplatz)。他在电视塔底层的咖啡馆购买了一杯拿铁,并与一名身份不明的男子交谈了五分钟。--09:15,Viktor 离开了亚历山大广场,乘出租车前往查理检查站(Checkpoint Charlie)。他的公文包里显然装着危险品。--09:45,冲突在 Friedrichstraße 车站的地下维修通道正式爆发。Elias 追上了 Bond(Viktor 的副手)。在一阵密集的交火后,Elias 成功击落了 Bond 手中的无线起爆器。-起爆器掉落在 3 号站台生锈的导轨上,距离带电轨道仅几厘米。--10:10,Viktor 出现在查理检查站附近的马路旁,他正准备与那里的地下组织会合。Sarah 的监视小组记录下了他与另一名接头人交换了一个蓝色的U盘。--10:30,真炸弹的最终位置被确认为威廉皇帝纪念教堂(Kaiser Wilhelm Memorial Church)。-Elias 在 10:28 冲上了教堂钟楼的最顶层平台,冒着巨大的生命危险剪断了那根决定生死的红线。--11:00,警方封锁了教堂周围的所有街道。Elias 悄无声息地从后门撤离,回到了勃兰登堡门附近的接头点。-"""--# ==================== 3. 运行提取 ====================--def run_demo():- print("=" * 80)- print("🕵️ AutoSpatioTemporalGraph 演示:冷战柏林行动")- print("=" * 80)-- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings()-- # 创建 AutoSpatioTemporalGraph 实例- graph = AutoSpatioTemporalGraph[SpyEntity, SpatioTemporalAction](- node_schema=SpyEntity,- edge_schema=SpatioTemporalAction,- node_key_extractor=lambda x: x.name,- # 基础边 Key: (谁, 做了什么, 对谁)- edge_key_extractor=lambda x: f"{x.source}|{x.action}|{x.target}",- # 分离提取:时间- time_in_edge_extractor=lambda x: x.time or "",- # 分离提取:空间- location_in_edge_extractor=lambda x: x.location or "",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder,- # 时空上下文注入- observation_time="2024-03-15",- observation_location="Brandenburg Gate, Berlin",- # 提示词微调- prompt_for_edge_extraction=(- "精确捕获行动发生的时间和地点。\n"- "特别是:\n"- "- 如果提到'这里',指 Brandenburg Gate\n"- "- 提取具体的车站名、建筑物名或房间号"- "### 已知的实体列表\n"- "{known_nodes}\n\n"- "## 源文本:\n"- "{source_text}\n\n"- ),- )-- print("\n⚙️ 执行时空感知提取流程...")- graph.feed_text(ST_NARRATIVE)-- # ==================== 4. 结果展示 ====================-- print("\n" + "=" * 80)- print(f"📊 提取结果统计:{len(graph.nodes)} 节点, {len(graph.edges)} 边")- print("=" * 80)-- print("\n📍 关键时空事件表 (Edges):")- # 按时间排序展示- sorted_edges = sorted(graph.edges, key=lambda x: (x.time or "9999", x.location or ""))- for i, edge in enumerate(sorted_edges, 1):- print(f"{i}. {edge}")-- # ==================== 5. 增强检索测试 ====================- print("\n" + "=" * 80)- print("🔍 多维时空检索测试")- print("=" * 80)- graph.build_index()-- print("\n📍 检索场景 1: 地点相关检索")- q1 = "在亚历山大广场(Alexanderplatz)发生了什么?"- _, edges1 = graph.search(q1, top_k_edges=3)- for e in edges1: print(f" ✅ {e}")-- print("\n💼 检索场景 2: 关键物品流转")- q2 = "Viktor 的公文包和蓝色 U 盘在哪里出现的?"- _, edges2 = graph.search(q2, top_k_edges=3)- for e in edges2: print(f" ✅ {e}")-- print("\n⏰ 检索场景 3: 时间同步检索")- q3 = "早上 09:00 这一刻,Elias 和 Viktor 分别在内存中留下了什么记录?"- _, edges3 = graph.search(q3, top_k_edges=5)- for e in edges3: print(f" ✅ {e}")-- # ==================== 6. 交互式分析 (Chat) ====================- print("\n" + "=" * 80)- print("💬 时空感知交互式情报分析 (Chat)")- print("=" * 80)-- chat_queries = [- "请按时间顺序列出 Elias 在这次行动中的所有地理位置变化轨迹。",- "描述 Viktor 在亚历山大广场和查理检查站的具体活动细节及其时间点。",- "对比分析 Elias 和 Viktor 在 09:00 时的动作,他们是否在同一地点?"- ]-- for cq in chat_queries:- print(f"\n👤 [高级分析师]: {cq}")- response = graph.chat(cq)- print(f"🤖 [情报分析系统]:\n{response.content}")-- # ==================== 7. 保存图谱 ====================- print("\n" + "=" * 80)- save_path = project_root / "temp" / "auto_spatio_temporal_demo"- graph.dump(save_path)- print(f"💾 完整时空知识图谱已持久化至: {save_path}")--if __name__ == "__main__":- run_demo()diff --git a/examples/types/temporal_graph_demo.py b/examples/types/temporal_graph_demo.pydeleted file mode 100644index 0bf4896..0000000--- a/examples/types/temporal_graph_demo.py+++ /dev/null@@ -1,337 +0,0 @@-"""-AutoTemporalGraph Demo: 谍战惊悚 - 柏林的48小时 🕵️♀️💣--场景背景:-在冷战时期的东柏林,一名CIA特工Elias必须在48小时内阻止恐怖分子的炸弹计划。-时间信息隐含在叙述中,通过人物对话、观察和行动序列自然出现。--难点:-1. 时间隐藏在对话中:"看了一眼手表"、"两分钟后"等自然表述-2. 并发事件:同一时间多个地点的不同行动-3. 时间精度:需要提取到分钟级别(HH:MM)-4. 实体合并:同一个人的不同代称需要被识别和合并--Usage:- python examples/types/temporal_graph_demo.py-"""--import sys-from pathlib import Path-from typing import Optional--project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))--from pydantic import BaseModel, Field-from langchain_openai import ChatOpenAI, OpenAIEmbeddings--from hyperextract.types import AutoTemporalGraph-from ontomem import MergeStrategy--import dotenv--dotenv.load_dotenv()--# ==================== 1. 定义时间感知知识结构 (Schema) ====================---class SpyEntity(BaseModel):- """谍战实体节点(特工、地点、组织等)"""-- name: str = Field(description="实体名称,例如特工ID、地点名、组织名")- category: str = Field(- description="实体类型:'特工', '地点', '组织', '物品', '事件'", default="未知"- )- description: str = Field(- description="实体的详细描述或身份背景", default=""- )-- def __repr__(self):- return f"🕵️ [{self.category}] {self.name} - {self.description}"---class TemporalAction(BaseModel):- """时间感知的行动边"""-- source: str = Field(description="行动发起者")- target: str = Field(description="行动对象(可以是人、地点或物品)")- action: str = Field(description="具体行动内容")- timestamp: Optional[str] = Field(- description="行动发生的具体时间点(YYYY-MM-DD HH:MM 格式,精确到分钟)", default=None- )- description: str = Field(- description="行动的详细描述、上下文或发生地点", default=""- )-- def __repr__(self):- time_str = f" @ {self.timestamp}" if self.timestamp else ""- desc_str = f" ({self.description})" if self.description else ""- return f"⚡ {self.source} --[{self.action}]--> {self.target}{time_str}{desc_str}"---# ==================== 2. 谍战叙述文本 ====================--SPY_THRILLER_NARRATIVE = """-【CIA 绝密档案:代号"日蚀"行动日志】-【日期:2024-03-15】-【地点:柏林】-【特工:Elias】--08:30,柏林的晨雾还未散去。特工 Elias 穿着一身灰色风衣,在勃兰登堡门附近的 Tiergarten 公园长椅上坐下。他在喂鸽子,看似悠闲,实则警惕地观察四周。--五分钟后(08:35),联络员 Sarah 骑着自行车经过,故意丢下了一份报纸。夹在报纸里的是一张加密纸条:"Viktor 在东区活动。第一枚炸弹已部署在 Friedrichstraße 车站。"--Elias 迅速离开。09:00,他在安全屋换装完毕,携带了干扰器和开锁工具。--09:15,Elias 抵达 Friedrichstraße 车站。车站人流如织。他在月台的自动售票机旁,发现了 Viktor 的助手 Bond。Bond 手提一个沉重的黑色公文包,神色慌张,正在频繁看表。--此时(09:15),在城市的另一端,废弃工厂的地下室里,Viktor 正在通过无线电向所有战斗小组下达指令:"不用等了,现在就启动倒计时。"--09:42,Bond 走向了车站深处的地下维修通道入口。Elias 悄无声息地跟了进去。通道里光线昏暗,只有滴水的声音。--突然,Bond 猛地转身,手里多了一把消音手枪。两人展开了近身搏斗。09:44,Bond 试图按下公文包上的红色按钮。--09:45,随着一声沉闷的枪响,Elias 击中了 Bond 的手腕,起爆器滑落到铁轨旁。此时公文包上的红色 LED 灯亮起,显示倒计时剩余时间:15分钟。--Elias 开始拆弹。这其实是一个复杂的诱饵装置。他在切断第一根红线时,发现电路逻辑不对。现在是 09:50。--"真的炸弹在哪里?" Elias 踩住 Bond 的伤口逼问。-Bond 疼得满头大汗,狞笑道:"你来不及了。真家伙在威廉皇帝纪念教堂。定时 10:30 引爆,那里现在全是游客。"--10:05,Elias 冲出车站,抢了一辆停在路边的杜卡迪摩托车。他在早高峰的车流中狂飙,连闯三个红灯。-与此同时(10:05),Sarah 收到了 Elias 的警报,带领 CIA 战术小队开始疏散教堂外围的人群。--10:15,Elias 冲进教堂。他在螺旋楼梯上被两名雇佣兵拦截。经过一番激烈的枪战,他解决了守卫,冲上钟楼顶部。--10:20,他在钟楼顶层发现了 Viktor。Viktor 手里抓着扩音器,似乎正准备发表某种宣言。看到 Elias,Viktor 拔出了匕首。--10:25,Elias 与 Viktor 在狭窄的钟楼平台展开肉搏。Viktor 试图将不知情的游客作为人质,Elias 抓住破绽,一记重拳将 Viktor 击倒,随后将其拷在栏杆上。--10:26,Elias 找到了藏在大钟底座下的生化炸弹。显示屏跳动着:00:03:59。-这不是普通的定时炸弹,还有倾斜传感器。--10:28,Elias 满头大汗。他通过耳机连接了 Sarah 传来的蓝图指示。"剪蓝线,还是黄线?"-"等等!" Sarah 在耳机里喊道,"图纸有更新,是红蓝相间的那根地线!"-Elias 深吸一口气,剪断了那根不起眼的细线。--计时器定格在 00:01:30。危机解除。--11:00,警方封锁线建立。Viktor 和 Bond 被押上囚车。Elias 在混乱的人群中压低帽檐,转身走进了一条小巷,消失在柏林的喧嚣中。-"""--# ==================== 3. 运行提取 ====================---def run_demo():- print("=" * 70)- print("🕵️ 柏林谍战行动 - 时间线分析模块启动")- print("=" * 70)-- # 初始化 LLM- print("\n🔧 初始化 LLM 和嵌入模型...")- llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)- embedder = OpenAIEmbeddings()-- # 创建 AutoTemporalGraph- print("📊 构建 AutoTemporalGraph 实例...\n")- graph = AutoTemporalGraph[SpyEntity, TemporalAction](- node_schema=SpyEntity,- edge_schema=TemporalAction,- # 1. 实体ID提取规则- node_key_extractor=lambda x: x.name,- # 2. 时间感知的行动唯一性规则(分离 edge key 和 time)- edge_key_extractor=lambda x: f"{x.source}|{x.action}|{x.target}",- time_in_edge_extractor=lambda x: x.timestamp or "",- # 3. 行动双方映射规则- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm,- embedder=embedder,- # 观察时间:故事背景- observation_time="2024-03-15",- # 两阶段提取- extraction_mode="two_stage",- # 智能合并:能识别"Elias" = "特工" 等- node_strategy_or_merger=MergeStrategy.LLM.BALANCED,- edge_strategy_or_merger=MergeStrategy.LLM.BALANCED,- # 针对谍战的特化提示词- prompt_for_node_extraction=(- "提取文本中的所有关键实体作为节点。包括:特工/人物、地点、组织、物品、事件。\n"- "关键要求:\n"- "1. 提取所有人名(Elias、Viktor、Bond、Katya、Sarah等)及其别称或代称\n"- "2. 提取所有地点(柏林、Black Swan酒吧、Charlottenburg Palace等)\n"- "3. 提取组织(CIA、KGB等)和物品(炸弹、枪、起爆器等)\n"- "4. 将代称和别名标准化为主要名称\n\n"- "## 源文本:\n"- "{source_text}"- ),- prompt_for_edge_extraction=(- "提取人物之间和与物品/地点之间的**时间感知的行动**。\n"- "关键要求:\n"- "1. 时间信息可能隐含在叙述中,如'看了一眼手表——秒针刚好划过十二点。现在是 18:42'\n"- "2. 相对时间表达如'两分钟后'、'十分钟过去了'需要根据前文推算具体时间\n"- "3. 并发事件('与此同时')需提取相同的时间戳\n"- "4. 提取精确到分钟级别(HH:MM格式),日期使用 YYYY-MM-DD\n"- "5. 在 description 字段中记录行动发生的地点或其他上下文信息\n"- "6. 如果时间不明确,填写空值而不是猜测"- "### 已知的实体列表\n"- "{known_nodes}\n\n"- "## 源文本:\n"- "{source_text}\n\n"- ),- chunk_size=2048,- )-- print(f"\n📖 正在加载谍战行动日志...")- print("=" * 70)- print(SPY_THRILLER_NARRATIVE[:300] + "...")- print("=" * 70)-- # 执行提取- print("\n⚙️ 执行两阶段时间感知提取...\n")- graph.feed_text(SPY_THRILLER_NARRATIVE)-- # ==================== 4. 时间线分析 ====================-- print("\n" + "=" * 70)- print(f"✅ 时间线分析完成!")- print("=" * 70)- print(f" 提取的实体: {len(graph.nodes)} 个")- print(f" 记录的行动事件: {len(graph.edges)} 个")-- print("\n" + "-" * 70)- print("🕵️ 实体信息 (Nodes)")- print("-" * 70)- for i, entity in enumerate(graph.nodes, 1):- print(f"{i}. {entity}")-- print("\n" + "-" * 70)- print("⚡ 时间感知行动时间表 (Edges)")- print("-" * 70)-- # 排序有时间的边- all_sorted_edges = sorted(graph.edges, key=lambda x: x.timestamp or "9999-12-31")- for i, action in enumerate(all_sorted_edges, 1):- print(f"{i}. {action}")-- # ==================== 5. 情报检索系统 ====================-- print("\n" + "=" * 70)- print("🔍 CIA 情报检索系统")- print("=" * 70)-- # 建立索引- graph.build_index()-- # 检索场景1: 找关键人物- q1 = "Elias做了什么?"- print(f"\n❓ Query: {q1}")- try:- nodes, edges = graph.search(q1, top_k_nodes=5, top_k_edges=5)- if edges:- for e in edges:- print(f" ⚡ {e}")- else:- print(" (无相关记录)")- except Exception as e:- print(f" (检索出错: {e})")-- # 检索场景2: 找地点- q2 = "发生在地铁站的事件有哪些?"- print(f"\n❓ Query: {q2}")- try:- nodes, edges = graph.search(q2, top_k_nodes=5, top_k_edges=5)- if edges:- for e in edges:- print(f" 📍 {e}")- else:- print(" (无相关记录)")- except Exception as e:- print(f" (检索出错: {e})")-- # 检索场景3: 找对抗- q3 = "Viktor和Elias之间的冲突"- print(f"\n❓ Query: {q3}")- try:- nodes, edges = graph.search(q3, top_k_nodes=5, top_k_edges=5)- if edges:- for e in edges:- print(f" 🔥 {e}")- else:- print(" (无相关记录)")- except Exception as e:- print(f" (检索出错: {e})")-- # 检索场景4: 找时间关键点- q4 = "下午的行动"- print(f"\n❓ Query: {q4}")- try:- nodes, edges = graph.search(q4, top_k_nodes=5, top_k_edges=5)- if edges:- for e in edges:- print(f" ⏰ {e}")- else:- print(" (无相关记录)")- except Exception as e:- print(f" (检索出错: {e})")-- # 检索场景5: 找组织- q5 = "CIA和KGB分别做了什么?"- print(f"\n❓ Query: {q5}")- try:- nodes, edges = graph.search(q5, top_k_nodes=5, top_k_edges=5)- if nodes:- print(f" 🏢 相关实体:")- for n in nodes:- print(f" - {n}")- if edges:- print(f" 📋 相关行动:")- for e in edges:- print(f" - {e}")- if not nodes and not edges:- print(" (无相关记录)")- except Exception as e:- print(f" (检索出错: {e})")-- # ==================== 6. 交互式情报问答 (Chat) ====================-- print("\n" + "=" * 70)- print("💬 正在切换至:CIA 交互式情报分析终端")- print("=" * 70)- print("系统提示: 您现在可以直接用自然语言询问关于行动的细节、动机或因果关系。\n")-- chat_queries = [- "Viktor的炸弹袭击计划最终是如何失败的?请按时间顺序列出导致他失败的关键转折点。",- "Elias在整个行动中使用了哪些交通工具?请结合时间点说明。",- "这次行动中,哪些人物的决策改变了事件的进程?"- ]-- for q in chat_queries:- print(f"👤 [高级分析师]: {q}")- print("🤖 [系统核心]: 正在检索关联情报节点并生成分析报告...")-- try:- # 使用 chat 接口进行 RAG 问答- response = graph.chat(q)- print(f"📄 [情报简报]:\n{response.content}\n")- except Exception as e:- print(f"⚠️ 系统响应异常: {e}\n")-- # ==================== 7. 保存知识图谱 ====================-- print("\n" + "=" * 70)- print("💾 保存时间感知知识图谱到本地...")- print("=" * 70)- save_path = project_root / "temp" / "auto_temporal_graph_demo"- try:- graph.dump(save_path)- print(f"✅ 图谱已保存到: {save_path}")- except Exception as e:- print(f"⚠️ 保存失败: {e}")-- print("\n" + "=" * 70)- print("✅ 演示完成!时间线已成功提取并分析。")- print("=" * 70)---if __name__ == "__main__":- run_demo()diff --git a/examples/zh/autotypes/graph_demo.py b/examples/zh/autotypes/graph_demo.pyindex 6704a43..add3576 100644--- a/examples/zh/autotypes/graph_demo.py+++ b/examples/zh/autotypes/graph_demo.py@@ -7,16 +7,21 @@python examples/zh/autotypes/graph_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class Entity(BaseModel):"""知识图谱中的实体"""@@ -31,14 +36,15 @@ class Relation(BaseModel):type: str = Field(description="关系类型:父亲/兄弟/师徒/朋友/政敌")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" 图谱提取示例")print("=" * 60)@@ -47,6 +53,9 @@ def main():graph = AutoGraph[Entity, Relation](node_schema=Entity,edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),llm_client=llm,embedder=embedder,)@@ -57,16 +66,15 @@ def main():graph.build_index()- for q in [- "苏轼的家庭关系有哪些?",- "苏轼与哪些人有过师徒关系?",- "苏轼创作了哪些著名作品?",- ]:- print(f"\n问题:{q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" 实体:{[n.name for n in nodes]}")- print(f" 关系:{[e.type for e in edges]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = graph.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: x.type)diff --git a/examples/zh/autotypes/hypergraph_demo.py b/examples/zh/autotypes/hypergraph_demo.pyindex 5dfbaa5..99ff208 100644--- a/examples/zh/autotypes/hypergraph_demo.py+++ b/examples/zh/autotypes/hypergraph_demo.py@@ -1,21 +1,26 @@"""超图示例 - 苏轼传记-使用 AutoHyperGraph 从文本中提取超关系。+使用 AutoHypergraph 从文本中提取超关系。使用方法:python examples/zh/autotypes/hypergraph_demo.py"""-import sysfrom pathlib import Path++from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv+from pydantic import BaseModel, Field-dotenv.load_dotenv()+from hyperextract.types import AutoHypergraph-from pydantic import BaseModel, Field-from hyperextract.types import AutoHyperGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"class Entity(BaseModel):@@ -31,22 +36,26 @@ class HyperEdge(BaseModel):type: str = Field(description="关系类型")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" 超图示例")print("=" * 60)print("提取多实体关系...")- graph = AutoHyperGraph[Entity, HyperEdge](+ graph = AutoHypergraph[Entity, HyperEdge](node_schema=Entity,edge_schema=HyperEdge,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.type}_{'_'.join(sorted(x.members))}",+ nodes_in_edge_extractor=lambda x: tuple(x.members),llm_client=llm,embedder=embedder,)@@ -61,11 +70,15 @@ def main():graph.build_index()- for q in ["乌台诗案", "创作", "贬谪"]:- print(f"\n查询:{q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" 找到 {len(edges)} 个相关超边")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = graph.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: x.type)diff --git a/examples/zh/autotypes/list_demo.py b/examples/zh/autotypes/list_demo.pyindex ec30c00..48d9bfd 100644--- a/examples/zh/autotypes/list_demo.py+++ b/examples/zh/autotypes/list_demo.py@@ -7,16 +7,21 @@ AutoList 示例 - 苏轼人生事件python examples/zh/autotypes/list_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract import AutoList+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class TimelineEvent(BaseModel):"""人生事件"""@@ -25,14 +30,15 @@ class TimelineEvent(BaseModel):description: str = Field(description="事件描述")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoList 示例 - 苏轼人生事件")print("=" * 60)@@ -54,12 +60,15 @@ def main():timeline.build_index()- for q in ["贬谪", "创作", "黄州"]:- print(f"\n查询:{q}")- results = timeline.search(q, top_k=3)- for r in results:- print(f" {r.year}:{r.title}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = timeline.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ timeline.show(item_label_extractor=lambda x: f"{x.year}:{x.title}")diff --git a/examples/zh/autotypes/model_demo.py b/examples/zh/autotypes/model_demo.pyindex 7292708..a84db1c 100644--- a/examples/zh/autotypes/model_demo.py+++ b/examples/zh/autotypes/model_demo.py@@ -7,17 +7,22 @@ AutoModel 示例 - 苏轼传记摘要python examples/zh/autotypes/model_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from typing import List, Optional+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field-from typing import List, Optional+from hyperextract import AutoModel+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class BiographySummary(BaseModel):"""传记摘要结构"""@@ -32,14 +37,15 @@ class BiographySummary(BaseModel):major_works: List[str] = Field(default_factory=list, description="代表作品")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoModel 示例 - 苏轼传记摘要")print("=" * 60)@@ -66,12 +72,15 @@ def main():model.build_index()- for q in ["作品", "黄州", "影响"]:- print(f"\n查询:{q}")- results = model.search(q, top_k=2)- for r in results:- print(f" {r}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = model.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ model.show(label_extractor=lambda x: x.title)diff --git a/examples/zh/autotypes/set_demo.py b/examples/zh/autotypes/set_demo.pyindex 68e35e8..4e463df 100644--- a/examples/zh/autotypes/set_demo.py+++ b/examples/zh/autotypes/set_demo.py@@ -7,16 +7,21 @@ AutoSet 示例 - 苏轼相关人物python examples/zh/autotypes/set_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract import AutoSet+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class Person(BaseModel):"""人物实体"""@@ -25,14 +30,15 @@ class Person(BaseModel):description: str = Field(description="简要描述", default="")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" AutoSet 示例 - 苏轼相关人物")print("=" * 60)@@ -60,12 +66,15 @@ def main():entities.build_index()- for q in ["苏洵", "王安石", "欧阳修"]:- print(f"\n查询:{q}")- results = entities.search(q, top_k=3)- for r in results:- print(f" {r.name}({r.category})")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = entities.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ entities.show(item_label_extractor=lambda x: x.name)diff --git a/examples/zh/autotypes/spatial_demo.py b/examples/zh/autotypes/spatial_demo.pyindex f336132..bbd4876 100644--- a/examples/zh/autotypes/spatial_demo.py+++ b/examples/zh/autotypes/spatial_demo.py@@ -7,16 +7,21 @@python examples/zh/autotypes/spatial_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoSpatialGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class Location(BaseModel):"""地点实体"""@@ -32,14 +37,15 @@ class SpatialRelation(BaseModel):relation: str = Field(description="空间关系类型")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" 空间图示例")print("=" * 60)@@ -48,6 +54,10 @@ def main():graph = AutoSpatialGraph[Location, SpatialRelation](node_schema=Location,edge_schema=SpatialRelation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ location_in_edge_extractor=lambda x: x.target,llm_client=llm,embedder=embedder,observation_location="北宋疆域",@@ -62,11 +72,15 @@ def main():graph.build_index()- for q in ["眉山", "黄州", "杭州"]:- print(f"\n查询:{q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" 地点:{[n.name for n in nodes]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = graph.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: x.relation)diff --git a/examples/zh/autotypes/spatio_temporal_demo.py b/examples/zh/autotypes/spatio_temporal_demo.pyindex 31fcfe9..67f8926 100644--- a/examples/zh/autotypes/spatio_temporal_demo.py+++ b/examples/zh/autotypes/spatio_temporal_demo.py@@ -7,17 +7,22 @@python examples/zh/autotypes/spatio_temporal_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from typing import Optional+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field-from typing import Optional+from hyperextract.types import AutoSpatioTemporalGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class Entity(BaseModel):"""实体节点"""@@ -34,14 +39,15 @@ class SpatioTemporalEvent(BaseModel):location: Optional[str] = Field(description="何地", default=None)-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" 时空图示例")print("=" * 60)@@ -50,6 +56,11 @@ def main():graph = AutoSpatioTemporalGraph[Entity, SpatioTemporalEvent](node_schema=Entity,edge_schema=SpatioTemporalEvent,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.action}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ time_in_edge_extractor=lambda x: x.time or "",+ location_in_edge_extractor=lambda x: x.location or "",llm_client=llm,embedder=embedder,observation_location="北宋",@@ -67,11 +78,15 @@ def main():graph.build_index()- for q in ["黄州", "杭州", "贬谪"]:- print(f"\n查询:{q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" 找到 {len(nodes)} 个实体,{len(edges)} 个事件")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = graph.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: f"{x.action}@{x.time}")diff --git a/examples/zh/autotypes/temporal_demo.py b/examples/zh/autotypes/temporal_demo.pyindex ed164c2..a16d6db 100644--- a/examples/zh/autotypes/temporal_demo.py+++ b/examples/zh/autotypes/temporal_demo.py@@ -7,16 +7,21 @@python examples/zh/autotypes/temporal_demo.py"""-import sysfrom pathlib import Path-from langchain_openai import ChatOpenAI, OpenAIEmbeddings-import dotenv--dotenv.load_dotenv()+from dotenv import load_dotenv+from langchain_openai import ChatOpenAI, OpenAIEmbeddingsfrom pydantic import BaseModel, Field+from hyperextract.types import AutoTemporalGraph+project_root = Path(__file__).resolve().parent.parent.parent++load_dotenv()++INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"+QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+class Entity(BaseModel):"""实体节点"""@@ -31,14 +36,15 @@ class TimelineEvent(BaseModel):entities: list[str] = Field(description="参与实体")-def main():+if __name__ == "__main__":+ with open(INPUT_FILE, encoding="utf-8") as f:+ text = f.read()+ with open(QUESTION_FILE, encoding="utf-8") as f:+ questions = [line.strip() for line in f if line.strip()]+llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)embedder = OpenAIEmbeddings(model="text-embedding-3-small")- input_file = Path(__file__).parent.parent.parent / "zh" / "sushi.md"- with open(input_file, "r", encoding="utf-8") as f:- text = f.read()-print("\n" + "=" * 60)print(" 时序图示例")print("=" * 60)@@ -47,6 +53,10 @@ def main():graph = AutoTemporalGraph[Entity, TimelineEvent](node_schema=Entity,edge_schema=TimelineEvent,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.time}-{x.event}",+ nodes_in_edge_extractor=lambda x: tuple(x.entities) if x.entities else (),+ time_in_edge_extractor=lambda x: x.time,llm_client=llm,embedder=embedder,)@@ -61,11 +71,15 @@ def main():graph.build_index()- for q in ["黄州", "贬谪", "乌台诗案"]:- print(f"\n查询:{q}")- nodes, edges = graph.search(q, top_k_nodes=3, top_k_edges=3)- print(f" 事件:{[e.time for e in edges]}")---if __name__ == "__main__":- main()+ print("-" * 60)+ print("问答")+ print("-" * 60)+ for q in questions:+ print(f"\n问: {q}")+ try:+ result = graph.chat(q)+ print(f"答: {result.content}")+ except Exception as e:+ print(f"错误: {e}")++ graph.show(node_label_extractor=lambda x: x.name, edge_label_extractor=lambda x: f"{x.event}@{x.time}")diff --git a/examples/zh/methods/atom_demo.py b/examples/zh/methods/atom_demo.pyindex 450c524..0654edd 100644--- a/examples/zh/methods/atom_demo.py+++ b/examples/zh/methods/atom_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/atom_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import Atom+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/cog_rag_demo.py b/examples/zh/methods/cog_rag_demo.pyindex af51574..c10f95f 100644--- a/examples/zh/methods/cog_rag_demo.py+++ b/examples/zh/methods/cog_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/cog_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Cog_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/graph_rag_demo.py b/examples/zh/methods/graph_rag_demo.pyindex 9512f76..61e279c 100644--- a/examples/zh/methods/graph_rag_demo.py+++ b/examples/zh/methods/graph_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/graph_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Graph_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/hyper_rag_demo.py b/examples/zh/methods/hyper_rag_demo.pyindex 175c56d..ba3a7f7 100644--- a/examples/zh/methods/hyper_rag_demo.py+++ b/examples/zh/methods/hyper_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/hyper_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Hyper_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/hypergraph_rag_demo.py b/examples/zh/methods/hypergraph_rag_demo.pyindex 4254885..162c598 100644--- a/examples/zh/methods/hypergraph_rag_demo.py+++ b/examples/zh/methods/hypergraph_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/hypergraph_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import HyperGraph_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/itext2kg_demo.py b/examples/zh/methods/itext2kg_demo.pyindex c2a2629..c63bad7 100644--- a/examples/zh/methods/itext2kg_demo.py+++ b/examples/zh/methods/itext2kg_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/itext2kg_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import iText2KG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/itext2kg_star_demo.py b/examples/zh/methods/itext2kg_star_demo.pyindex 886e73a..9cafc4d 100644--- a/examples/zh/methods/itext2kg_star_demo.py+++ b/examples/zh/methods/itext2kg_star_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/itext2kg_star_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import iText2KG_Star+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/kg_gen_demo.py b/examples/zh/methods/kg_gen_demo.pyindex 51fad2b..b4fb027 100644--- a/examples/zh/methods/kg_gen_demo.py+++ b/examples/zh/methods/kg_gen_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/kg_gen_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.typical import KG_Gen+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/methods/light_rag_demo.py b/examples/zh/methods/light_rag_demo.pyindex 749c803..e77bccb 100644--- a/examples/zh/methods/light_rag_demo.py+++ b/examples/zh/methods/light_rag_demo.py@@ -7,19 +7,19 @@ Usage:python examples/zh/methods/light_rag_demo.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))from dotenv import load_dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract.methods.rag import Light_RAG+project_root = Path(__file__).resolve().parent.parent.parent+load_dotenv()-INPUT_FILE = project_root / "examples" / "zh" / "sushi.md"-QUESTION_FILE = project_root / "examples" / "zh" / "sushi_question.md"+INPUT_FILE = project_root / "zh" / "sushi.md"+QUESTION_FILE = project_root / "zh" / "sushi_question.md"if __name__ == "__main__":with open(INPUT_FILE, encoding="utf-8") as f:diff --git a/examples/zh/sushi_question.md b/examples/zh/sushi_question.mdindex 820fca5..4a7b890 100644--- a/examples/zh/sushi_question.md+++ b/examples/zh/sushi_question.md@@ -1,3 +1,3 @@-1. 苏轼的家庭关系和师友有哪些?-2. 苏轼的仕途经历如何影响了他的文学创作?-3. 苏轼被贬谪期间创作了哪些著名作品?+- 苏轼的家庭关系和师友有哪些?+- 苏轼的仕途经历如何影响了他的文学创作?+- 苏轼被贬谪期间创作了哪些著名作品?diff --git a/examples/zh/templates/finance_template.py b/examples/zh/templates/finance_template.pyindex d567401..0c7d888 100644--- a/examples/zh/templates/finance_template.py+++ b/examples/zh/templates/finance_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/finance_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/zh/templates/general_template.py b/examples/zh/templates/general_template.pyindex 36c2d44..8148e2d 100644--- a/examples/zh/templates/general_template.py+++ b/examples/zh/templates/general_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/general_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/zh/templates/industry_template.py b/examples/zh/templates/industry_template.pyindex e545e54..90f152b 100644--- a/examples/zh/templates/industry_template.py+++ b/examples/zh/templates/industry_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/industry_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/zh/templates/legal_template.py b/examples/zh/templates/legal_template.pyindex f0b8df0..97984dd 100644--- a/examples/zh/templates/legal_template.py+++ b/examples/zh/templates/legal_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/legal_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/zh/templates/list_templates.py b/examples/zh/templates/list_templates.pyindex 185c399..02cf0c4 100644--- a/examples/zh/templates/list_templates.py+++ b/examples/zh/templates/list_templates.py@@ -5,12 +5,10 @@ Usage:python examples/templates/list_templates.py"""-import sysimport yamlfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))+project_root = Path(__file__).resolve().parent.parent.parent.parentdef list_templates():diff --git a/examples/zh/templates/medicine_template.py b/examples/zh/templates/medicine_template.pyindex 9e18183..5253477 100644--- a/examples/zh/templates/medicine_template.py+++ b/examples/zh/templates/medicine_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/medicine_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/examples/zh/templates/tcm_template.py b/examples/zh/templates/tcm_template.pyindex ab3e0ff..b5788f2 100644--- a/examples/zh/templates/tcm_template.py+++ b/examples/zh/templates/tcm_template.py@@ -5,16 +5,14 @@ Usage:python examples/templates/tcm_template.py"""-import sysfrom pathlib import Path-project_root = Path(__file__).resolve().parent.parent.parent-sys.path.append(str(project_root))-+import dotenvfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings+from hyperextract import Template-import dotenv+project_root = Path(__file__).resolve().parent.parent.parent.parentdotenv.load_dotenv()diff --git a/hyperextract/methods/typical/atom.py b/hyperextract/methods/typical/atom.pyindex 5b3d349..f86e5c7 100644--- a/hyperextract/methods/typical/atom.py+++ b/hyperextract/methods/typical/atom.py@@ -87,14 +87,14 @@ class EdgeSchema(BaseModel):description=("A time or interval indicating when this relationship begins or is active. ""Resolve relative temporal expressions based on the observation_time:\n"- " - 'today' 鈫?exact observation_time\n"- " - 'yesterday' 鈫?observation_time minus 1 day\n"- " - 'this week' 鈫?Monday of observation_time's week\n"- " - 'last week' 鈫?Monday of the week before observation_time\n"- " - 'this month' 鈫?first day of observation_time's month\n"- " - 'last month' 鈫?first day of the month before observation_time\n"- " - 'this year' 鈫?January 1st of observation_time's year\n"- " - 'last year' 鈫?January 1st of the year before observation_time\n"+ " - 'today' → exact observation_time\n"+ " - 'yesterday' → observation_time minus 1 day\n"+ " - 'this week' → Monday of observation_time's week\n"+ " - 'last week' → Monday of the week before observation_time\n"+ " - 'this month' → first day of observation_time's month\n"+ " - 'last month' → first day of the month before observation_time\n"+ " - 'this year' → January 1st of observation_time's year\n"+ " - 'last year' → January 1st of the year before observation_time\n""Keep explicit dates as-is (e.g., '18-06-2024'). ""For example, if 'Yassir became CEO from 2023', then t_start=['01-01-2023']. ""This can be a single year, a date, or a resolved relative reference. "@@ -106,9 +106,9 @@ class EdgeSchema(BaseModel):description=("A time or interval indicating when this relationship ceases to hold. ""Resolve relative temporal expressions based on the observation_time using the same rules as t_start:\n"- " - 'today' 鈫?exact observation_time\n"- " - 'yesterday' 鈫?observation_time minus 1 day\n"- " - 'this week' 鈫?Monday of observation_time's week\n"+ " - 'today' → exact observation_time\n"+ " - 'yesterday' → observation_time minus 1 day\n"+ " - 'this week' → Monday of observation_time's week\n"" - etc. (same resolution rules as t_start)\n""Keep explicit dates as-is. ""For example, if 'Yassir left his position in 2025', then t_end=['01-01-2025']. "@@ -203,7 +203,7 @@ Given an input paragraph and an `observation_time`, generate a list of all disti"""# Atom FACToid prompt ends, edge prompt begins-Atom_EDGE_EXTRACTION_PROMPT = """+Atom_EDGE_EXTRACTION_PROMPT = f"""Observation Date: {observation_time}You are a precise knowledge extraction engine designed to distill unstructured text into a structured Knowledge Graph.@@ -218,7 +218,7 @@ Extract a list of relationships where each relationship consists of:5. **Time End (`t_end`)**: Strategies for when the relation ended.6. **Evidence (`atomic_facts`)**: The exact source sentences/facts supporting this edge.-### 馃摑 Guidelines+### 🎯 Guidelines#### 1. Entity & Relation Standards- **Entities**: Extract precise names (e.g., "Apple Inc.", "Steve Jobs"). Assign accurate semantic labels (e.g., "Organization", "Person").@@ -241,7 +241,7 @@ Extract a list of relationships where each relationship consists of:- **NO Paraphrasing**: Do not summarize or rewrite the source text in this field.- If a relationship is derived from multiple facts, include all distinct facts in the list.-### 馃�� Few-Shot Examples+### 📝 Few-Shot Examples**Input**: "Michel served as CFO at Acme Corp from 2019 to 2021. He was hired by Beta Inc in 2021, but left that role in 2023."**Output**:@@ -261,15 +261,12 @@ Extract a list of relationships where each relationship consists of:**Output**:- (John Doe, has_status, Married, t_start=["2026-02-26"], t_end=[], atomic_facts=["John Doe's marriage is happening on 26-02-2026."])-### 馃殌 Directives+### ⚡ Directives- Extract strictly what is in the text. Do not hallucinate external facts.- Use the **Observation Date** as the anchor for all relative temporal expressions.- Maintain consistency: The same entity name should be used across multiple relations.- Ensure `atomic_facts` contains exact copies of the source text strings.-### Known Nodes:-{known_nodes}-### Source Text:{source_text}"""@@ -385,7 +382,7 @@ class Atom(AutoGraph[NodeSchema, EdgeSchema]):# Inject customized promptsprompt_for_edge_extraction=Atom_EDGE_EXTRACTION_PROMPT,# Configure deduplication strategy- node_strategy_or_merger=MergeStrategy.KEEP_EXISTING,+ node_strategy_or_merger=MergeStrategy.LLM.BALANCED,edge_strategy_or_merger=edge_merger,# Optimize indexing: only index name fieldnode_fields_for_index=["name", "label"],@@ -561,7 +558,7 @@ class Atom(AutoGraph[NodeSchema, EdgeSchema]):The updated Atom instance with matched nodes and updated edges."""logger.info(- f"馃攧 Starting node matching and edge update (threshold={threshold})..."+ f"🚀 Starting node matching and edge update (threshold={threshold})...")nodes, edges = self.nodes, self.edgesdiff --git a/hyperextract/methods/typical/itext2kg_star.py b/hyperextract/methods/typical/itext2kg_star.pyindex 89b7a44..7ed3359 100644--- a/hyperextract/methods/typical/itext2kg_star.py+++ b/hyperextract/methods/typical/itext2kg_star.py@@ -274,7 +274,7 @@ class iText2KG_Star(AutoGraph[NodeSchema, EdgeSchema]):The updated iText2KG_Star instance with matched nodes and updated edges."""logger.info(- f"馃攧 Starting node matching and edge update (threshold={threshold})..."+ f"🚀 Starting node matching and edge update (threshold={threshold})...")nodes, edges = self.nodes, self.edgesdiff --git a/hyperextract/methods/typical/kg_gen.py b/hyperextract/methods/typical/kg_gen.pyindex 3718ed0..2cc3376 100644--- a/hyperextract/methods/typical/kg_gen.py+++ b/hyperextract/methods/typical/kg_gen.py@@ -251,7 +251,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):new_instance._set_data_state(graph_data)logger.info(- f"鉁?Deduplication Complete: "+ f"✅ Deduplication Complete: "f"Nodes {original_node_count} -> {len(new_instance.nodes)}, "f"Edges {original_edge_count} -> {len(new_instance.edges)}")@@ -271,7 +271,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):self (modified in-place)"""logger.info(- f"馃攧 Starting in-place SemHash deduplication (threshold={threshold})..."+ f"🚀 Starting in-place SemHash deduplication (threshold={threshold})...")graph_data = self.data.model_copy()@@ -286,7 +286,7 @@ class KG_Gen(AutoGraph[NodeSchema, EdgeSchema]):self._set_data_state(graph_data)logger.info(- f"鉁?In-place deduplication complete: "+ f"✅ In-place deduplication complete: "f"Nodes {original_node_count} -> {len(self.nodes)}, "f"Edges {original_edge_count} -> {len(self.edges)}")diff --git a/tests/conftest.py b/tests/conftest.pyindex 6eb0a63..00411f7 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -7,6 +7,14 @@ import pytestfrom dotenv import load_dotenv+def pytest_configure(config):+ """Register custom markers."""+ config.addinivalue_line(+ "markers",+ "integration: marks tests as integration tests (requires OPENAI_API_KEY env var)",+ )++# Load .env file if it existsdef _load_env():"""Load environment variables from .env file."""diff --git a/tests/fixtures/__init__.py b/tests/fixtures/__init__.pynew file mode 100644index 0000000..b2e0948--- /dev/null+++ b/tests/fixtures/__init__.py@@ -0,0 +1,39 @@+"""Shared test fixtures and schemas for hyperextract tests."""++from .schemas import (+ PersonSchema,+ CompanySchema,+ EntitySchema,+ RelationSchema,+ KeywordSchema,+ ProductSchema,+ BiographySchema,+ EarningsSummarySchema,+)+from .data import (+ SHORT_TEXT,+ LONG_TEXT,+ MULTIPLE_PERSONS_TEXT,+ MULTIPLE_COMPANIES_TEXT,+ GRAPH_TEXT,+ KEYWORD_TEXT,+ PERSON_LIST_TEXT,+)++__all__ = [+ "PersonSchema",+ "CompanySchema",+ "EntitySchema",+ "RelationSchema",+ "KeywordSchema",+ "ProductSchema",+ "BiographySchema",+ "EarningsSummarySchema",+ "SHORT_TEXT",+ "LONG_TEXT",+ "MULTIPLE_PERSONS_TEXT",+ "MULTIPLE_COMPANIES_TEXT",+ "GRAPH_TEXT",+ "KEYWORD_TEXT",+ "PERSON_LIST_TEXT",+]diff --git a/tests/fixtures/data.py b/tests/fixtures/data.pynew file mode 100644index 0000000..92d9d14--- /dev/null+++ b/tests/fixtures/data.py@@ -0,0 +1,48 @@+"""Shared test data for hyperextract tests."""++SHORT_TEXT = """John Smith is a software engineer at TechCorp. He has 10 years of experience in software development."""++LONG_TEXT = """+Tesla, Inc. was founded in 2003 by Martin Eberhard and Marc Tarpenning.+Elon Musk joined in 2004 and became the company's CEO. The company is headquartered in Palo Alto, California.+Tesla designs, develops, manufactures, and sells electric vehicles, battery energy storage, and solar products.+The company's mission is to accelerate the world's transition to sustainable energy.+In 2023, Tesla delivered over 1.8 million vehicles globally.+"""++MULTIPLE_PERSONS_TEXT = """+Alice Johnson works as a data scientist at Google. She graduated from MIT with a PhD in Machine Learning.+Bob Williams is a product manager at Meta. He has 8 years of experience in tech product management.+Carol Davis is a UX designer at Apple. She studied at Stanford and has won several design awards.+David Chen is a software engineer at Amazon. He specializes in distributed systems and cloud computing.+"""++MULTIPLE_COMPANIES_TEXT = """+Apple Inc. is a technology company headquartered in Cupertino, California. It was founded in 1976 by Steve Jobs, Steve Wozniak, and Ronald Wayne. Apple designs and manufactures smartphones, computers, tablets, and wearable devices.++Microsoft Corporation is a software company based in Redmond, Washington. It was founded in 1975 by Bill Gates and Paul Allen. Microsoft develops operating systems, productivity software, and cloud services.++Google LLC is a technology company headquartered in Mountain View, California. It was founded in 1998 by Larry Page and Sergey Brin. Google specializes in internet-related services and products.+"""++GRAPH_TEXT = """+Apple was founded by Steve Jobs in Cupertino, California. Steve Jobs later became CEO of Apple.+Apple acquired NeXT in 1996, which brought Steve Jobs back to Apple.+Tim Cook became Apple's CEO after Steve Jobs stepped down in 2011.+Apple collaborated with IBM in 2014 to compete against enterprise mobile computing.+"""++KEYWORD_TEXT = """+Machine learning is a subset of artificial intelligence that enables systems to learn from data.+Deep learning uses neural networks with multiple layers to model complex patterns.+Natural language processing focuses on enabling computers to understand human language.+Computer vision allows machines to interpret and understand visual information from the world.+"""++PERSON_LIST_TEXT = """+John is a software engineer with 5 years of experience at Google.+Mary works as a product manager at Facebook.+Tom is a data scientist at Amazon specializing in recommendation systems.+Sarah is a UX designer at Apple with expertise in mobile interfaces.+Michael is a DevOps engineer at Microsoft.+"""diff --git a/tests/fixtures/schemas.py b/tests/fixtures/schemas.pynew file mode 100644index 0000000..fd227a5--- /dev/null+++ b/tests/fixtures/schemas.py@@ -0,0 +1,77 @@+"""Shared Pydantic schemas for testing."""++from typing import Optional, List+from pydantic import BaseModel, Field+++class PersonSchema(BaseModel):+ """Schema for person entities."""++ name: str = Field(description="Full name of the person")+ age: Optional[int] = Field(default=None, description="Age of the person")+ occupation: Optional[str] = Field(default=None, description="Job or profession")+ email: Optional[str] = Field(default=None, description="Email address")+++class CompanySchema(BaseModel):+ """Schema for company entities."""++ name: str = Field(description="Company name")+ industry: Optional[str] = Field(default=None, description="Industry sector")+ founded_year: Optional[int] = Field(default=None, description="Year founded")+ headquarters: Optional[str] = Field(default=None, description="Headquarters location")+++class EntitySchema(BaseModel):+ """Schema for graph entity/node extraction."""++ name: str = Field(description="Entity name")+ type: str = Field(description="Entity type (e.g., PERSON, ORGANIZATION, LOCATION)")+ properties: dict = Field(default_factory=dict, description="Additional properties")+++class RelationSchema(BaseModel):+ """Schema for graph relation/edge extraction."""++ source: str = Field(description="Source entity name")+ target: str = Field(description="Target entity name")+ relation_type: str = Field(description="Type of relationship")+ description: Optional[str] = Field(default=None, description="Relationship description")+++class KeywordSchema(BaseModel):+ """Schema for keyword extraction."""++ term: str = Field(description="Keyword or key phrase")+ category: Optional[str] = Field(default=None, description="Category or domain")+ frequency: Optional[int] = Field(default=None, description="Frequency count")+++class ProductSchema(BaseModel):+ """Schema for product extraction."""++ name: str = Field(description="Product name")+ price: Optional[float] = Field(default=None, description="Product price")+ description: Optional[str] = Field(default=None, description="Product description")+ features: List[str] = Field(default_factory=list, description="List of features")+++class BiographySchema(BaseModel):+ """Schema for biography/summary extraction."""++ name: str = Field(description="Person's full name")+ birth_date: Optional[str] = Field(default=None, description="Date of birth")+ death_date: Optional[str] = Field(default=None, description="Date of death")+ nationality: Optional[str] = Field(default=None, description="Nationality")+ occupation: List[str] = Field(default_factory=list, description="Professions/occupations")+ summary: str = Field(description="Brief biography summary")+++class EarningsSummarySchema(BaseModel):+ """Schema for financial earnings summary."""++ company_name: str = Field(description="Company name")+ quarter: str = Field(description="Fiscal quarter (e.g., Q1 2024)")+ revenue: Optional[float] = Field(default=None, description="Revenue in millions")+ net_income: Optional[float] = Field(default=None, description="Net income in millions")+ earnings_per_share: Optional[float] = Field(default=None, description="EPS")diff --git a/tests/mocks.py b/tests/mocks.pyindex 9a1b1de..e449104 100644--- a/tests/mocks.py+++ b/tests/mocks.py@@ -1,6 +1,7 @@"""Mock implementations for LLM and Embeddings for testing without API keys."""-from typing import Any, List, Optional, Type, get_origin, get_args+from typing import Any, List, Optional, Type, Union, get_origin, get_args, Dict+from enum import Enumfrom pydantic import BaseModel, create_modelfrom langchain_core.language_models.chat_models import BaseChatModelfrom langchain_core.messages import BaseMessage, AIMessage@@ -25,7 +26,6 @@ class MockEmbeddings(Embeddings):def _hash_to_vector(self, text: str) -> List[float]:"""Convert text to deterministic vector of given dimension."""hash_val = hash(text)- # Create deterministic values based on hashreturn [(hash_val + i) % 1000 / 1000.0 for i in range(self.dim)]@@ -55,6 +55,19 @@ class MockStructuredRunnable(RunnableSerializable):"""Generate mock instances for batch input."""return [self.invoke(inp, config) for inp in inputs]+ def _get_field_info(self, field_type: Type) -> tuple:+ """Get the base type and whether it's Optional."""+ origin = get_origin(field_type)++ if origin is Union:+ args = get_args(field_type)+ non_none_args = [arg for arg in args if arg is not type(None)]+ if len(non_none_args) == 1 and type(None) in args:+ return non_none_args[0], True+ return field_type, False++ return field_type, False+def _generate_mock_instance(self, model_cls: Type[BaseModel]) -> BaseModel:"""Recursively generate mock instance matching schema."""data = {}@@ -62,46 +75,64 @@ class MockStructuredRunnable(RunnableSerializable):for field_name, field_info in model_cls.model_fields.items():field_type = field_info.annotation- # Handle Optional typesorigin = get_origin(field_type)- if origin is Optional.__class__ or (- hasattr(field_type, "__args__") and type(None) in get_args(field_type)- ):- # Extract the actual type from Optional- args = get_args(field_type)- field_type = args[0] if args else str+ is_optional = False- origin = get_origin(field_type)+ if origin is Union:+ base_type, is_optional = self._get_field_info(field_type)+ field_type = base_type+ origin = get_origin(field_type)++ default = field_info.default+ has_default = (+ default is not None+ or field_info.default_factory is not None+ )- # Handle List types- if origin is list:- args = get_args(field_type)- if args and issubclass(args[0], BaseModel):- # List of Pydantic models- data[field_name] = [self._generate_mock_instance(args[0])]- else:- # List of primitives or empty- data[field_name] = []-- # Handle Pydantic model fields- elif isinstance(field_type, type) and issubclass(field_type, BaseModel):- data[field_name] = self._generate_mock_instance(field_type)-- # Handle primitive types- elif field_type == str:- data[field_name] = f"mock_{field_name}"- elif field_type == int:- data[field_name] = 1- elif field_type == float:- data[field_name] = 1.0- elif field_type == bool:- data[field_name] = True- else:- # Default to None for unknown types+ if is_optional and not has_default:data[field_name] = None+ continue++ value = self._generate_value_for_type(field_type, origin, field_name)+ data[field_name] = valuereturn model_cls(**data)+ def _generate_value_for_type(+ self, field_type: Type, origin: Any, field_name: str+ ) -> Any:+ """Generate a mock value for a given field type."""+ if origin is list:+ args = get_args(field_type)+ if args and issubclass(args[0], BaseModel):+ return [self._generate_mock_instance(args[0])]+ return []++ if origin is dict:+ args = get_args(field_type)+ key_type = args[0] if len(args) > 0 else str+ value_type = args[1] if len(args) > 1 else Any+ return {}++ if isinstance(field_type, type) and issubclass(field_type, Enum):+ return list(field_type)[0] if len(field_type) > 0 else None++ if isinstance(field_type, type) and issubclass(field_type, BaseModel):+ return self._generate_mock_instance(field_type)++ if field_type == str:+ return f"mock_{field_name}"+ elif field_type == int:+ return 1+ elif field_type == float:+ return 1.0+ elif field_type == bool:+ return True+ elif field_type == Any:+ return f"mock_{field_name}"+ else:+ return None+class MockChatModel(BaseChatModel):"""Mock Chat Model for testing without API calls."""@@ -118,10 +149,15 @@ class MockChatModel(BaseChatModel):**kwargs: Any,) -> Any:"""Generate mock response."""- # Return a simple mock message- return type("ChatResult", (), {"generations": [type("Generation", (), {"message": AIMessage(content="mock response")})]})()-- def with_structured_output(self, schema: Type[BaseModel], **kwargs: Any) -> RunnableSerializable:+ return type("ChatResult", (), {+ "generations": [+ type("Generation", (), {"message": AIMessage(content="mock response")})+ ]+ })()++ def with_structured_output(+ self, schema: Type[BaseModel], **kwargs: Any+ ) -> RunnableSerializable:"""Return a MockStructuredRunnable that generates mock data matching the schema.This simulates LangChain's with_structured_output behavior.diff --git a/tests/template_engine/__init__.py b/tests/template_engine/__init__.pynew file mode 100644index 0000000..824d943--- /dev/null+++ b/tests/template_engine/__init__.py@@ -0,0 +1 @@+"""Tests for hyperextract template_engine module."""diff --git a/tests/template_engine/test_factory.py b/tests/template_engine/test_factory.pynew file mode 100644index 0000000..1cc5d47--- /dev/null+++ b/tests/template_engine/test_factory.py@@ -0,0 +1,114 @@+"""Unit tests for TemplateFactory."""++import pytest+from pathlib import Path++from hyperextract.utils.template_engine import TemplateFactory, Gallery+++class TestTemplateFactoryCreate:+ """Test cases for TemplateFactory create methods."""++ def test_get_existing_template(self):+ """Test getting an existing template from gallery."""+ template = Gallery.get("general/model")++ assert template is not None+ assert template.name == "model"+ assert template.type in ["model", "list", "set", "graph"]++ def test_get_nonexistent_template(self):+ """Test getting a nonexistent template returns None."""+ template = Gallery.get("nonexistent/template")++ assert template is None+++class TestTemplateFactoryCreateMethod:+ """Test cases for TemplateFactory create method."""++ def test_create_with_language(self, llm_client, embedder):+ """Test create() with language parameter."""+ result = TemplateFactory.create(+ source="general/model",+ language="en",+ llm_client=llm_client,+ embedder=embedder,+ )++ assert result is not None+ assert result.metadata.get("lang") == "en"++ def test_create_without_source_raises(self, llm_client, embedder):+ """Test that create() raises error without source."""+ with pytest.raises((ValueError, TypeError)):+ TemplateFactory.create(+ llm_client=llm_client,+ embedder=embedder,+ )++ def test_create_method_template(self, llm_client, embedder):+ """Test creating a method template."""+ result = TemplateFactory.create(+ source="method/light_rag",+ llm_client=llm_client,+ embedder=embedder,+ )++ assert result is not None+++class TestTemplateFactoryCreateAllTypes:+ """Test cases for creating all AutoType types."""++ def test_create_model_type(self, llm_client, embedder):+ """Test create() with model type template."""+ result = TemplateFactory.create(+ source="general/model",+ language="en",+ llm_client=llm_client,+ embedder=embedder,+ )++ from hyperextract.types import AutoModel+ assert isinstance(result, AutoModel)+ assert result.metadata.get("type") == "model"++ def test_create_list_type(self, llm_client, embedder):+ """Test create() with list type template."""+ result = TemplateFactory.create(+ source="general/list",+ language="en",+ llm_client=llm_client,+ embedder=embedder,+ )++ from hyperextract.types import AutoList+ assert isinstance(result, AutoList)+ assert result.metadata.get("type") == "list"++ def test_create_set_type(self, llm_client, embedder):+ """Test create() with set type template."""+ result = TemplateFactory.create(+ source="general/set",+ language="en",+ llm_client=llm_client,+ embedder=embedder,+ )++ from hyperextract.types import AutoSet+ assert isinstance(result, AutoSet)+ assert result.metadata.get("type") == "set"++ def test_create_graph_type(self, llm_client, embedder):+ """Test create() with graph type template."""+ result = TemplateFactory.create(+ source="general/graph",+ language="en",+ llm_client=llm_client,+ embedder=embedder,+ )++ from hyperextract.types import AutoGraph+ assert isinstance(result, AutoGraph)+ assert result.metadata.get("type") == "graph"diff --git a/tests/template_engine/test_gallery.py b/tests/template_engine/test_gallery.pynew file mode 100644index 0000000..5532c69--- /dev/null+++ b/tests/template_engine/test_gallery.py@@ -0,0 +1,147 @@+"""Unit tests for Gallery."""++import pytest++from hyperextract.utils.template_engine import Gallery+++class TestGalleryGet:+ """Test cases for Gallery.get() method."""++ def test_get_existing_template_by_path(self):+ """Test getting an existing template by full path."""+ template = Gallery.get("general/model")++ assert template is not None+ assert hasattr(template, "name")+ assert hasattr(template, "type")++ def test_get_existing_template_by_name(self):+ """Test getting an existing template by name only."""+ template = Gallery.get("model")++ assert template is not None++ def test_get_nonexistent_template(self):+ """Test getting a nonexistent template."""+ template = Gallery.get("nonexistent/template/that/does/not/exist")++ assert template is None++ def test_get_template_preserves_config(self):+ """Test that getting a template preserves its configuration."""+ template = Gallery.get("general/model")++ assert template is not None+ assert hasattr(template, "type")+ assert hasattr(template, "output")+ assert hasattr(template, "guideline")+++class TestGalleryList:+ """Test cases for Gallery.list() method."""++ def test_list_all_templates(self):+ """Test listing all templates."""+ templates = Gallery.list()++ assert isinstance(templates, dict)+ assert len(templates) > 0++ def test_filter_by_type_model(self):+ """Test filtering templates by model type."""+ templates = Gallery.list(filter_by_type="model")++ assert isinstance(templates, dict)+ for template in templates.values():+ assert template.type == "model"++ def test_filter_by_type_list(self):+ """Test filtering templates by list type."""+ templates = Gallery.list(filter_by_type="list")++ assert isinstance(templates, dict)+ for template in templates.values():+ assert template.type == "list"++ def test_filter_by_type_set(self):+ """Test filtering templates by set type."""+ templates = Gallery.list(filter_by_type="set")++ assert isinstance(templates, dict)+ for template in templates.values():+ assert template.type == "set"++ def test_filter_by_type_graph(self):+ """Test filtering templates by graph type."""+ templates = Gallery.list(filter_by_type="graph")++ assert isinstance(templates, dict)+ for template in templates.values():+ assert template.type == "graph"++ def test_filter_by_language_en(self):+ """Test filtering templates by English language."""+ templates = Gallery.list(filter_by_language="en")++ assert isinstance(templates, dict)++ def test_filter_by_language_zh(self):+ """Test filtering templates by Chinese language."""+ templates = Gallery.list(filter_by_language="zh")++ assert isinstance(templates, dict)++ def test_filter_by_query(self):+ """Test filtering templates by query."""+ templates = Gallery.list(filter_by_query="model")++ assert isinstance(templates, dict)++ def test_filter_by_type_and_language(self):+ """Test filtering by both type and language."""+ templates = Gallery.list(filter_by_type="model", filter_by_language="en")++ assert isinstance(templates, dict)+ for template in templates.values():+ assert template.type == "model"++ def test_filter_by_tag(self):+ """Test filtering templates by tag."""+ templates = Gallery.list(filter_by_tag="general")++ assert isinstance(templates, dict)+++class TestGalleryMultiple:+ """Test cases for Gallery with multiple templates."""++ def test_multiple_template_types(self):+ """Test that multiple template types are available."""+ model_templates = Gallery.list(filter_by_type="model")+ list_templates = Gallery.list(filter_by_type="list")+ set_templates = Gallery.list(filter_by_type="set")+ graph_templates = Gallery.list(filter_by_type="graph")++ total_templates = (+ len(model_templates)+ + len(list_templates)+ + len(set_templates)+ + len(graph_templates)+ )++ all_templates = Gallery.list()+ assert total_templates <= len(all_templates)++ def test_preserves_original_config(self):+ """Test that template config is preserved when filtering."""+ template = Gallery.get("general/model")++ if template is None:+ pytest.skip("Template not available")++ templates = Gallery.list(filter_by_type=template.type)++ assert len(templates) > 0+ for key, t in templates.items():+ assert t.type == template.typediff --git a/tests/template_engine/test_parser_loader.py b/tests/template_engine/test_parser_loader.pynew file mode 100644index 0000000..1456fe4--- /dev/null+++ b/tests/template_engine/test_parser_loader.py@@ -0,0 +1,30 @@+"""Unit tests for template parsers - loader."""++import pytest+import tempfile+import yaml+from pathlib import Path++from hyperextract.utils.template_engine.parsers import (+ load_template,+)+++class TestLoadTemplate:+ """Test cases for load_template function."""++ def test_load_nonexistent_file_raises(self):+ """Test that loading a nonexistent file raises FileNotFoundError."""+ with pytest.raises(FileNotFoundError):+ load_template("/nonexistent/path/to/template.yaml")++ def test_load_existing_template(self):+ """Test loading an existing template from the gallery."""+ from hyperextract.utils.template_engine import Gallery++ template = Gallery.get("general/model")++ assert template is not None+ assert hasattr(template, "name")+ assert hasattr(template, "type")+ assert hasattr(template, "output")diff --git a/tests/template_engine/test_parser_output.py b/tests/template_engine/test_parser_output.pynew file mode 100644index 0000000..128b899--- /dev/null+++ b/tests/template_engine/test_parser_output.py@@ -0,0 +1,47 @@+"""Unit tests for template parsers - output."""++import pytest+import tempfile+import yaml+from pathlib import Path+from pydantic import BaseModel++from hyperextract.utils.template_engine.parsers import (+ load_template,+)+++class TestParseOutputBasic:+ """Basic test cases for parse_output function."""++ def test_template_has_output(self):+ """Test that loaded template has output configuration."""+ from hyperextract.utils.template_engine import Gallery++ template = Gallery.get("general/model")++ assert template is not None+ assert hasattr(template, "output")++ def test_template_output_has_fields(self):+ """Test that template output has fields."""+ from hyperextract.utils.template_engine import Gallery++ template = Gallery.get("general/model")++ assert template is not None+ assert hasattr(template, "output")+ assert hasattr(template.output, "fields")+++class TestParseIdentifiers:+ """Test cases for parse_identifiers function."""++ def test_template_has_identifiers(self):+ """Test that loaded template has identifiers configuration."""+ from hyperextract.utils.template_engine import Gallery++ template = Gallery.get("general/set")++ assert template is not None+ assert hasattr(template, "identifiers")diff --git a/tests/test_data/en/finance/annual_report_sample.md b/tests/test_data/en/finance/annual_report_sample.mdnew file mode 100644index 0000000..1519a4d--- /dev/null+++ b/tests/test_data/en/finance/annual_report_sample.md@@ -0,0 +1,239 @@+# Galaxy Intelligent Technology Co., Ltd.++## 2025 Annual Report++**Stock Code**: 688567+**Stock Abbreviation**: Galaxy Intelligence+**Listing Exchange**: Shanghai Stock Exchange STAR Market++---++## Company Overview++| Item | Content |+|---|---|+| Company Name (Chinese) | Galaxy Intelligent Technology Co., Ltd. |+| Company Name (English) | Galaxy Intelligent Technology Co., Ltd. |+| Registered Address | Building A, Technology Park, No. 88 Gaoxin South First Road, Nanshan District, Shenzhen |+| Legal Representative | Michael Chen |+| Board Secretary | Sarah Lin |+| Registered Capital | RMB 420 million |+| Date of Incorporation | June 15, 2012 |+| Unified Social Credit Code | 91440300MA5DXXXXX0 |++---++## Section 1: Important Notices, Contents and Definitions++### 1. Important Notices++The Board of Directors, President and other senior management personnel of the Company ensure that the annual report content is true, accurate and complete, and there are no false records, misleading statements or material omissions. They shall bear individual and joint legal responsibility for this.++The Company's 2025 profit distribution plan is: based on the total share capital of 420,000,000 shares, to distribute cash dividends of RMB 2.80 per 10 shares (including tax) to all shareholders, totaling RMB 117,600,000, accounting for 32.5% of the net profit attributable to shareholders of the listed company. No bonus shares will be distributed, and no capital reserve will be converted into share capital.++---++## Section 2: Company Introduction and Key Financial Indicators++### 2.1 Company Introduction++Galaxy Intelligent Technology Co., Ltd. is a high-tech enterprise specializing in industrial artificial intelligence and intelligent manufacturing solutions. The Company primarily provides AI-based intelligent inspection, predictive maintenance and production optimization software platforms for manufacturing, energy, transportation and other industries. The Company's self-developed "Galaxy Core" AI platform has served over 1,280 enterprise customers, covering 28 provinces and cities nationwide and 8 overseas countries and regions.++### 2.2 Main Business Segments++The Company's business is divided into three major segments:++**1. Intelligent Inspection and Quality Control (45% of revenue)**++The Company's flagship product "Galaxy Vision" utilizes deep learning and computer vision technology to provide Automatic Optical Inspection (AOI) solutions for semiconductor, new energy vehicle, consumer electronics and other industries. During the reporting period, the intelligent inspection business achieved operating revenue of RMB 1.785 billion, a year-over-year increase of 28.5%.++**2. Predictive Maintenance (38% of revenue)**++The "Galaxy Predict" product achieves equipment fault prediction and maintenance optimization through industrial IoT sensor data collection and AI analysis. During the reporting period, it achieved operating revenue of RMB 1.508 billion, a year-over-year increase of 35.2%.++**3. Energy Management and Carbon Neutrality (17% of revenue)**++The energy management business segment established in 2023 provides factory energy consumption optimization and carbon emission management solutions. During the reporting period, it achieved operating revenue of RMB 675 million, a year-over-year increase of 125.8%.++### 2.3 Key Financial Data and Indicators++#### Consolidated Income Statement (Unit: RMB 10,000)++| Item | 2025 | 2024 | 2023 |+|---|---|---|---|+| **Operating Revenue** | 396,800 | 310,500 | 245,200 |+| Operating Costs | 221,400 | 182,650 | 147,120 |+| **Gross Profit** | 175,400 | 127,850 | 98,080 |+| R&D Expenses | 71,420 | 55,890 | 44,140 |+| Selling Expenses | 39,680 | 34,155 | 29,420 |+| Administrative Expenses | 18,920 | 15,580 | 12,260 |+| Financial Expenses | 2,840 | 3,210 | 2,180 |+| **Operating Profit** | 42,540 | 19,015 | 10,080 |+| Total Profit | 41,280 | 18,220 | 9,650 |+| Income Tax Expense | 6,192 | 2,733 | 1,448 |+| **Net Profit Attributable to Parent Company Shareholders** | 36,180 | 15,660 | 8,365 |+| **Basic EPS (RMB)** | 0.86 | 0.37 | 0.20 |+| **Diluted EPS (RMB)** | 0.85 | 0.37 | 0.20 |++#### Consolidated Balance Sheet (Unit: RMB 10,000)++| Item | December 31, 2025 | December 31, 2024 |+|---|---|---|+| **Current Assets** | | |+| Cash and Cash Equivalents | 85,420 | 52,680 |+| Accounts Receivable | 68,350 | 48,920 |+| Inventories | 12,450 | 9,820 |+| Other Current Assets | 5,680 | 4,210 |+| **Total Current Assets** | 171,900 | 115,630 |+| **Non-current Assets** | | |+| Fixed Assets | 45,820 | 38,560 |+| Intangible Assets | 18,920 | 15,420 |+| Goodwill | 22,500 | 12,500 |+| Right-of-use Assets | 8,650 | 7,210 |+| Other Non-current Assets | 4,180 | 3,560 |+| **Total Non-current Assets** | 100,070 | 77,250 |+| **Total Assets** | 271,970 | 192,880 |+| | | |+| **Current Liabilities** | | |+| Accounts Payable | 42,180 | 32,450 |+| Contract Liabilities | 15,620 | 12,380 |+| Short-term Borrowings | 8,500 | 12,000 |+| Other Current Liabilities | 6,840 | 5,210 |+| **Total Current Liabilities** | 73,140 | 62,040 |+| **Non-current Liabilities** | | |+| Long-term Borrowings | 18,000 | 15,000 |+| Lease Liabilities | 7,210 | 5,850 |+| Deferred Tax Liabilities | 1,680 | 980 |+| **Total Non-current Liabilities** | 26,890 | 21,830 |+| **Total Liabilities** | 100,030 | 83,870 |+| | | |+| **Owners' Equity** | | |+| Share Capital | 42,000 | 42,000 |+| Capital Reserve | 68,500 | 32,500 |+| Surplus Reserve | 8,540 | 6,250 |+| Undistributed Profits | 53,400 | 28,260 |+| **Total Equity Attributable to Parent Company Owners** | 172,440 | 109,010 |+| **Total Liabilities and Owners' Equity** | 271,970 | 192,880 |++#### Consolidated Cash Flow Statement (Unit: RMB 10,000)++| Item | 2025 | 2024 |+|---|---|---|+| Net Cash Flows from Operating Activities | 42,850 | 28,420 |+| Net Cash Flows from Investing Activities | (28,500) | (22,180) |+| Net Cash Flows from Financing Activities | 18,450 | (5,620) |+| Net Increase in Cash and Cash Equivalents | 32,800 | 620 |++#### Key Financial Indicators++| Indicator | 2025 | 2024 |+|---|---|---|+| Gross Margin | 44.2% | 41.2% |+| Net Profit Margin | 9.1% | 5.0% |+| Weighted Average ROE | 25.6% | 14.8% |+| R&D Expenses as % of Revenue | 18.0% | 18.0% |+| Revenue Growth Rate | 27.8% | 26.6% |+| Asset-liability Ratio | 36.8% | 43.5% |+| Current Ratio | 2.35 | 1.86 |+| Operating Cash Flow / Net Profit | 118.4% | 181.5% |++---++## Section 3: Management Discussion and Analysis++### 3.1 Discussion and Analysis of Business Operations During the Reporting Period++In 2025, the Company achieved operating revenue of RMB 3.968 billion, a year-over-year increase of 27.8%; net profit attributable to shareholders of the listed company was RMB 362 million, a year-over-year increase of 130.8%. The significant growth in business performance was mainly driven by the following factors:++**1. Strong Industry Demand**++Driven by the national "14th Five-Year Plan" Intelligent Manufacturing Development Plan and manufacturing digital transformation policies, the industrial AI software market maintained high-speed growth. According to data from CCID Research Institute, China's industrial AI market reached RMB 48.5 billion in 2025, a year-over-year increase of 32.5%. Leveraging its technology leadership and brand accumulation, the Company's market share increased from 6.8% in 2024 to 8.2%.++**2. Significant Results from Key Customer Strategy**++During the reporting period, the number of customers with annual revenue exceeding RMB 5 million increased from 85 in 2024 to 128, a year-over-year increase of 50.6%. The combined revenue from the top 10 customers accounted for 38.5%, a decrease from 42.3% in the previous year, indicating continuous optimization of customer concentration. New signings of industry leaders such as CATL, BYD and SMIC further validated the Company's product competitiveness in high-end manufacturing.++Among newly signed orders, the average contract value increased from RMB 6.8 million in 2024 to RMB 9.2 million, a growth of 35.3%. Customer renewal rate reached 92%, and revenue renewal rate (including upsells) reached 125%, demonstrating strong product stickiness and continuous cross-selling capability.++**3. Gross Margin Improvement**++The Company's comprehensive gross margin increased from 41.2% in 2024 to 44.2%, mainly due to: (1) the increase in software subscription revenue proportion from 52% to 65%, with subscription model gross margin of approximately 82%, higher than the 55% of project-based implementation model; (2) improved AI model inference efficiency reduced cloud computing resource consumption, with unit computing costs decreasing by approximately 25%; (3) economies of scale improved the personnel efficiency of the customer success team.++**4. Continuous Increase in R&D Investment**++During the reporting period, R&D investment was RMB 714 million, a year-over-year increase of 27.8%, accounting for 18% of operating revenue. The total number of R&D personnel reached 685, accounting for 42% of total employees. Main R&D directions include:+- Galaxy Core 3.0 Platform Upgrade: Introduction of LLM-driven natural language interaction interface+- Cross-modal AI Fusion: Integration of visual, acoustic, vibration and other multi-source data+- Edge Computing Solutions: Edge deployment solutions for offline and low-latency scenarios+- Carbon Footprint Tracking: Carbon emission accounting engine compliant with ISO 14064 and GHG Protocol++### 3.2 Key Business Risks During the Reporting Period++**1. Technology Iteration Risk**++Artificial intelligence technology is evolving rapidly, and the rapid advancement of Large Language Models (LLM) and generative AI may change the technological paradigm of industrial AI applications. If the Company fails to timely track technology development trends and complete product iteration and upgrades, it may face the risk of technological backwardness. The Company will continue to increase R&D investment, planning to maintain R&D expense ratio at 18%-20% in 2026.++**2. Intensifying Market Competition Risk**++The industrial AI market has attracted many competitors, including international giants such as Siemens and Schneider Electric, as well as domestic leading enterprises such as Hikvision and iFlytek. Some competitors have advantages in capital, channels and brand. Intensifying competition may lead to product price pressure and increased customer acquisition costs.++**3. Accounts Receivable Risk**++As the business scale expands, the Company's accounts receivable balance continues to grow. As of December 31, 2025, accounts receivable balance was RMB 684 million, accounting for 17.2% of operating revenue. Although the aging is mainly concentrated within 1 year (accounting for 82%), large accounts receivable still carry collection risk. The Company has accrued bad debt provisions of RMB 34.2 million, with bad debt provision accrual ratio of 5%.++**4. Talent Loss Risk**++The Company's core technical team is key to its competitiveness. During the reporting period, the core technical personnel loss rate was 8.5%, lower than the industry average of 12%. The Company has implemented an equity incentive plan, granting 8.5 million stock options to 168 core employees to enhance talent retention.++---++## Section 4: Corporate Governance++### 4.1 Directors, Supervisors and Senior Management++| Name | Position | Age | Start Date |+|---|---|---|---|+| Michael Chen | Chairman and CEO | 48 | June 2012 |+| David Wang | VP and CTO | 45 | March 2015 |+| Linda Liu | VP and CFO | 42 | September 2018 |+| James Zhang | VP and COO | 44 | April 2019 |+| Dr. Wei Li | Chief AI Scientist | 40 | July 2020 |+| Emily Zhao | Board Secretary | 38 | February 2021 |++### 4.2 Shareholder Information++#### Top 10 Shareholders (as of December 31, 2025)++| Shareholder Name | Shares Held (10,000) | Shareholding Ratio | Shareholder Type |+|---|---|---|---|+| Michael Chen | 8,500 | 20.24% | Founder |+| David Wang | 4,200 | 10.00% | Founder |+| Shenzhen Innovation Investment Group | 2,800 | 6.67% | State-owned Shareholder |+| Sequoia Capital China | 2,100 | 5.00% | Financial Investor |+| Hillhouse Capital | 1,680 | 4.00% | Financial Investor |+| National Integrated Circuit Industry Fund | 1,260 | 3.00% | State Fund |+| Tencent Industrial Investment Fund | 1,050 | 2.50% | Strategic Investor |+| China Life Insurance | 840 | 2.00% | Financial Investor |+| HKSCC (Stock Connect) | 630 | 1.50% | Foreign Investor |+| Bosera Funds | 420 | 1.00% | Public Fund |++---++## Section 5: Significant Events++### 5.1 Major Acquisitions++In August 2025, the Company acquired 100% equity of Shanghai CloudInspect Technology Co., Ltd. for cash consideration of RMB 225 million. CloudInspect is a leading domestic semiconductor wafer inspection AI software provider with over 28 semiconductor industry customers. After the acquisition, the Company's market share in the semiconductor inspection field increased from 5.2% to 8.5%.++The acquisition generated goodwill of RMB 125 million. The Company has conducted impairment tests on goodwill in accordance with Accounting Standards for Business Enterprises, and no impairment occurred in 2025.++### 5.2 Litigation and Arbitration++During the reporting period, the Company had no significant litigation or arbitration matters. The patent infringement dispute with Shenzhen SmartInspect Technology Co., Ltd. disclosed in 2024 was settled in September 2024, and the Company did not bear any compensation liability.++---++**This report was reviewed and approved by the Company's Board of Directors on April 20, 2026.**++Chairman Signature: /s/ Michael Chen+Date: April 20, 2026diff --git a/tests/test_data/en/finance/earnings_call_transcript_sample.md b/tests/test_data/en/finance/earnings_call_transcript_sample.mdnew file mode 100644index 0000000..f3a062a--- /dev/null+++ b/tests/test_data/en/finance/earnings_call_transcript_sample.md@@ -0,0 +1,238 @@+# Galaxy Intelligent Technology Co., Ltd.++## Q4 and Full Year 2025 Earnings Conference Call Transcript++| | |+|---|---|+| **Date** | January 15, 2026 |+| **Time** | 3:00 PM (Beijing Time) |+| **Participants** | Michael Chen - Chairman and CEO; Linda Liu - VP and CFO; David Wang - VP and CTO; Emily Zhao - Board Secretary |+| **Moderator** | Emily Zhao - Board Secretary |++---++## I. Management Remarks++### Emily Zhao — Board Secretary++Good afternoon, investors and analysts. Welcome to Galaxy Intelligent Technology's Q4 and Full Year 2025 Earnings Conference Call. Before we begin, please note that any forward-looking statements regarding future outlook and performance expectations mentioned during this call involve uncertainties, and actual results may differ from expectations.++Now, I would like to invite Mr. Michael Chen to introduce the Company's full-year business performance.++---++### Michael Chen — Chairman and CEO++Thank you, Emily. Good afternoon, investor friends.++2025 is a milestone year in Galaxy Intelligent's development history. The Company achieved operating revenue of RMB 3.968 billion, a year-over-year increase of 27.8%; net profit attributable to parent company shareholders was RMB 362 million, a year-over-year increase of 130.8%. This achievement was made possible by the efforts of all employees and the general trend of digital transformation in China's manufacturing industry.++I would like to emphasize three key aspects:++**First, breakthrough progress in the key customer strategy.** During the reporting period, we signed multiple industry-leading customers, including CATL, BYD and SMIC. The acquisition of these benchmark customers not only brought direct revenue contribution but, more importantly, validated our product competitiveness in the high-end manufacturing sector. The number of customers with annual revenue exceeding RMB 5 million increased from 85 to 128, a growth of 50.6%.++**Second, outstanding performance in the new business segment.** The energy management and carbon neutrality business achieved revenue of RMB 675 million, a year-over-year increase of 125.8%, making it the fastest-growing segment of the Company. With the advancement of carbon peak and carbon neutrality goals and the restart of the CCER market, we expect this segment to maintain growth of over 40% in 2026.++**Third, continuous deepening of technological innovation.** We have made significant progress in integrating large models with industrial AI. The Galaxy Core 3.0 platform introduced an LLM-driven natural language interaction interface, enabling frontline engineers to directly query equipment status and issue maintenance commands through natural language. This feature has been deployed and verified at 85 customer sites with positive feedback.++Now, let me invite CFO Linda Liu to present the financial data.++---++### Linda Liu — VP and CFO++Thank you, Mr. Chen. Let me present the key financial data for Q4 and the full year.++#### Q4 Performance++Q4 achieved operating revenue of RMB 1.258 billion, a year-over-year increase of 32.5% and a quarter-over-quarter increase of 18.2%. Q4 is typically our peak season, benefiting from customers' year-end budget execution.++Breaking down by business segment:+- **Intelligent Inspection and Quality Control**: Q4 revenue of RMB 568 million, year-over-year increase of 35.2%, with semiconductor industry customers contributing the main growth+- **Predictive Maintenance**: Q4 revenue of RMB 452 million, year-over-year increase of 38.5%, with rapid growth from new energy vehicle industry customers+- **Energy Management and Carbon Neutrality**: Q4 revenue of RMB 238 million, year-over-year increase of 145.8%, benefiting from the recognition of multiple large carbon management projects++#### Full Year Core Financial Indicators++| Indicator | 2025 | 2024 | YoY Change |+|---|---|---|---|+| Operating Revenue | RMB 3.968B | RMB 3.105B | +27.8% |+| Gross Margin | 44.2% | 41.2% | +3 ppts |+| Net Profit | RMB 362M | RMB 157M | +130.8% |+| Operating Cash Flow | RMB 428M | RMB 284M | +50.7% |+| R&D Investment | RMB 714M | RMB 559M | +27.7% |+| R&D as % of Revenue | 18.0% | 18.0% | Flat |++The gross margin improved by 3 percentage points to 44.2%, mainly due to three reasons: First, the proportion of software subscription revenue increased to 65%, with subscription model gross margin of approximately 82%, significantly higher than the 55% of project-based model; second, AI inference efficiency optimization reduced unit computing costs by approximately 25%; third, economies of scale improved personnel efficiency.++Operating cash flow increased by 50.7% year-over-year to RMB 428 million, mainly driven by net profit growth and increase in advance payments. Free cash flow was RMB 385 million, a year-over-year increase of 62.8%.++#### 2026 Performance Guidance++The Company provides the following business outlook for 2026:+- **Operating Revenue**: Expected RMB 5.0-5.5 billion, year-over-year increase of 26%-38%+- **Gross Margin**: Expected to maintain at 44%-46%+- **R&D Investment Ratio**: Expected to maintain at 18%-20%+- **Net Profit Margin**: Expected to improve to 11%-13%++Now, let me invite CTO David Wang to present the technology progress.++---++### David Wang — VP and CTO++Thank you, CFO Liu. Let me present the Company's latest technology progress.++**Galaxy Core 3.0 Platform Launch**++In September 2025, we officially launched the Galaxy Core 3.0 platform, the most important product upgrade in the Company's history. Key breakthroughs in version 3.0 include:++1. **LLM Integration**: First integration of LLM capabilities into industrial scenarios, supporting natural language queries and command issuance. Customers can directly obtain analysis results through natural language such as "query the equipment failure rate of Workshop C this month."++2. **Multi-modal Fusion**: Integrated visual, acoustic, vibration and other multi-sensor data to achieve comprehensive perception of equipment status.++3. **Edge Computing Capability**: Launched a lightweight edge deployment solution supporting offline deployment at industrial sites to meet data security requirements.++**Customer Expansion**++After the launch of Galaxy Core 3.0, we received strong market feedback. In Q4 2025, we newly signed 42 customers, with 20 upgrading to version 3.0. Benchmark customers include CATL's battery production lines, BYD's welding workshops, and SMIC's wafer fabs.++**2026 Technology Roadmap**++In 2026, we will continue to increase R&D investment, with key directions including:+- **Galaxy Core 3.5 Version**: Enhance multi-agent collaboration capabilities, supporting automated decision-making in complex industrial scenarios+- **Industrial Foundation Models**: Train vertical models for specific industrial scenarios to further reduce deployment costs+- **Overseas Version Products**: Develop international versions supporting multiple languages to serve overseas customers++---++## II. Q&A Session++### Q1: Analyst Liu from CICC++Congratulations on the excellent results. My question is about the energy management business. Could you please detail the growth drivers for this business? Also, what is the gross margin level for this business?++### Michael Chen++Thank you, Analyst Liu. The energy management business is a new business segment we established in 2023. In 2025, it achieved excellent growth with revenue of RMB 675 million, a year-over-year increase of 125.8%.++The growth drivers mainly include:++1. **Policy Push**: Under the dual carbon goals, more and more enterprises have demands for carbon emission management. Especially with the expected restart of the CCER market in 2025, it brings a large amount of incremental demand.++2. **Improved Product Maturity**: After more than two years of development, our products have become relatively mature and can meet customers' actual needs.++3. **Benchmark Customer Effect**: We signed multiple industry-leading customers, and the demonstration effect of these customers drove follow-up purchases from small and medium customers.++Regarding gross margin, the current gross margin of the energy management business is approximately 38%, lower than the Company's overall gross margin of 44.2%. The main reason is that this business requires more on-site implementation work. However, as the proportion of subscription revenue increases, we expect this business's gross margin to gradually improve.++---++### Q2: Analyst Wang from Huatai Securities++I have two questions. First, regarding the increase in subscription revenue proportion, the Company mentioned it reached 65% in 2025 and the target is 80% in 2027. What is the path for this? Second, accounts receivable has also grown relatively quickly. Could you explain the reasons and collection plans?++### Linda Liu++Thank you, Analyst Wang. Let me answer these two questions.++Regarding subscription revenue proportion:++In 2025, we reached 65% subscription revenue proportion, an increase of 13 percentage points from 52% in 2024. The improvement path mainly includes:++1. **New Contracts**: In 2025 new contracts, the subscription model accounted for 75%+2. **Existing Customer Conversion**: Some original project-based customers were converted to subscription model+3. **Upsell Sales**: Upsells to existing customers basically adopt the subscription model++We are very confident in achieving the 80% target by 2027. With the promotion of Galaxy Core 3.0, we expect the subscription proportion of new contracts to further increase.++Regarding accounts receivable:++Indeed, the accounts receivable balance at the end of 2025 was RMB 684 million, a significant increase from RMB 489 million at the end of 2024. The main reasons are:+1. Revenue scale increased by 27.8%, and accounts receivable grew synchronously+2. Q4 revenue proportion increased, and Q4 collections typically come in the following year++Regarding collection, we have taken the following measures:+1. Strengthened accounts receivable management, establishing a tiered early warning mechanism+2. Implemented special follow-up for key customers+3. Optimized contract terms to increase advance payment proportion++We expect accounts receivable turnover days to gradually improve in 2026.++---++### Q3: Institutional Investor++My question is about the competitive landscape. How does the Company view competition with leaders like Hikvision? What are the Company's differentiated technology advantages?++### David Wang++Thank you for this investor's question.++Regarding competition, we have always believed that healthy competition benefits the overall development of the industry. The industrial AI market is very large, and different companies have different positioning and advantages.++Our differentiated advantages mainly include:++1. **Vertical Depth**: We focus on industrial scenarios and have a deeper understanding of manufacturing. Our products are not general AI, but professional AI that truly solves industrial problems.++2. **Platform Capability**: The Galaxy platform supports multiple industrial application scenarios, allowing customers to solve various problems on one platform. This is more attractive than single-point solutions.++3. **Data Accumulation**: We have served over 1,280 customers and accumulated a large amount of industrial data, which helps our models continuously optimize.++4. **Customer Stickiness**: Our 92% customer renewal rate and 125% revenue renewal rate demonstrate customers' recognition of our products.++Of course, we will continue to monitor competitors' movements and continuously improve our competitiveness.++---++### Q4: Individual Investor++What is the Company's R&D investment plan for 2026? Will the R&D expense ratio increase or decrease?++### Linda Liu++Thank you for this investor's question.++In 2025, our R&D investment was RMB 714 million, with R&D expense ratio of 18%, flat compared to 2024.++In 2026, we expect R&D investment to be between RMB 900 million and RMB 1 billion, with R&D expense ratio maintaining at approximately 18%.++Key investment directions include:+1. R&D of Galaxy Core 3.5 version+2. Training of industrial foundation models+3. Development of overseas version products+4. Continuous optimization of core algorithms++We believe R&D investment is the foundation for maintaining competitiveness and will continue to maintain a high investment level.++---++## III. Closing Remarks++### Michael Chen++Thank you for the questions from investors and analysts.++2025 is a milestone year for Galaxy Intelligent. We achieved high-speed growth in revenue and profit, made breakthroughs in both the number and quality of customers, and continuously improved product competitiveness.++In 2026, we will continue to adhere to the two core strategies of "key customer strategy" and "technology innovation-driven growth" to promote the Company's sustainable and healthy development. We are confident in achieving the 2026 business goals and creating greater value for investors.++Thank you for your participation. See you at the next earnings conference call!++---++## IV. Supplementary Information++### Contact Information++- Investor Hotline: 0755-XXXXXXX+- Email: [email protected]+- Website: www.galaxytech.com++### Next Earnings Call Date++The Q1 2026 results are expected to be released in April 2026, with the earnings conference call scheduled for late April.++---++*This transcript is for reference only and does not constitute investment advice.*diff --git a/tests/test_data/en/finance/equity_research_report_sample.md b/tests/test_data/en/finance/equity_research_report_sample.mdnew file mode 100644index 0000000..123d109--- /dev/null+++ b/tests/test_data/en/finance/equity_research_report_sample.md@@ -0,0 +1,98 @@+# Deep Research Report++## Galaxy Intelligent Technology (688567.SH) — Industrial AI Leader, Large Models Empower New Growth++| | |+|---|---|+| **Analyst** | Michael Zhang, CFA |+| **Research Firm** | Huatai Securities Research |+| **Publication Date** | November 15, 2025 |+| **Rating** | Buy (Maintained) |+| **Previous Rating** | Buy |+| **Target Price** | RMB 85.60 |+| **Previous Target Price** | RMB 68.00 |+| **Latest Close Price** | RMB 62.50 (November 14, 2025) |+| **Upside Potential** | 37.0% |+| **Market Cap** | RMB 36.05 billion |+| **Industry** | Software & IT Services |++---++## Investment Thesis++We maintain our "Buy" rating on Galaxy Intelligent and raise the target price to RMB 85.60 (from RMB 68.00), based on three core investment logics:++1. **Sustained Industrial AI Momentum**: Driven by the "14th Five-Year Plan" Intelligent Manufacturing Development Plan and manufacturing digital transformation policies, China's industrial AI market reached RMB 48.5 billion in 2025, with 2025-2028 CAGR expected at 25%. As the leading industrial AI software company, the Company's market share increased from 6.8% to 8.2%, continuously consolidating its industry leadership position.++2. **Accelerating Subscription Transformation Unlocking Profitability**: The Company's software subscription revenue proportion increased from 52% in 2024 to 65%, with a target of 80% by 2027. Subscription model gross margin is approximately 82% (vs. project-based 55%), driving comprehensive gross margin from 41.2% to 44.2%. We expect net margin to further improve from 9.1% to 12%-15% in 2026-2027.++3. **Carbon Neutrality Business Opens Second Growth Curve**: The energy management and carbon neutrality business achieved 125.8% revenue growth in 2025. Driven by dual carbon policies and EU CBAM, it is expected to maintain growth above 40% in 2026. This business currently accounts for only 17% of Company revenue, with enormous penetration potential.++---++## Catalysts Analysis++### Near-term Catalysts (0-6 months)++- **2025 Annual Report Disclosure** (expected April 2026): We expect full-year revenue of RMB 3.97 billion, exceeding market consensus of RMB 3.85 billion. Net profit of RMB 362 million, exceeding consensus of RMB 320 million. Earnings beat could drive valuation re-rating.+- **Galaxy Core 3.0 Mass Rollout**: Expected to add 200-300 customer upgrades to version 3.0 in H1 2026. LLM integration features will enhance product differentiation.+- **CCER Market Restart**: National CCER (China Certified Emission Reduction) market expected to officially restart in Q1 2026, which will directly drive demand for Company carbon management software.++### Mid-term Catalysts (6-18 months)++- **Semiconductor Industry Recovery**: Global semiconductor inventory adjustment expected to complete in H1 2026. The Company's acquired CloudInspect Technology will fully benefit from wafer fab capacity utilization recovery. Semiconductor-related revenue expected to reach RMB 350-450 million in 2026.+- **Overseas Market Expansion**: Company plans to establish regional headquarters in Singapore in 2026, targeting overseas revenue proportion of 10% by 2027. Cooperation with FANUC is expected to materialize in 2026.++### Long-term Catalysts (18+ months)++- **Industrial Foundation Models**: Company is developing vertical foundation models for industrial scenarios, expected to launch in 2027-2028, which could significantly reduce customization costs for AI deployment and expand addressable market.+- **M&A Integration**: Company has RMB 850 million in cash on hand with low leverage, enabling continued acquisition integration in industrial AI segments.++---++## Risk Analysis++### Downside Risks++1. **Macroeconomic Downturn Risk** (Probability: Medium, Timeframe: 6-12 months): If manufacturing PMI continues below 50, customers may delay IT investments. We estimate that for every 1 percentage point decline in GDP, Company revenue growth could slow by 3-5 percentage points.++2. **Intensifying Competition Risk** (Probability: Medium, Timeframe: 12-24 months): Competitors such as Siemens MindSphere and Huawei FusionPlant are accelerating promotions. Hikvision may pose competitive pressure on the Company's intelligent inspection business leveraging its advantages in visual technology. If Company's market share declines by 1 percentage point, corresponding revenue impact is approximately RMB 40 million.++3. **Accounts Receivable Risk** (Probability: Low, Timeframe: Ongoing): Company accounts receivable balance is RMB 684 million, accounting for 17.2% of revenue. If some key customers face payment difficulties, it may generate bad debt losses.++4. **Talent Loss Risk** (Probability: Low-Medium, Timeframe: Ongoing): AI industry talent competition is intense. Loss of core technical personnel may affect product R&D progress. Currently core team attrition rate is 8.5%, below industry average of 12%, but requires continuous attention.++5. **Goodwill Impairment Risk** (Probability: Low, Timeframe: 12-24 months): Acquisition of CloudInspect generated goodwill of RMB 125 million. If CloudInspect business falls short of expectations, there is goodwill impairment risk.++---++## Earnings Forecast++### Revenue Forecast (Unit: RMB Billion)++| Segment | 2024A | 2025E | 2026E | 2027E |+|---|---|---|---|---|+| Intelligent Inspection & QC | 1.242 | 1.785 | 2.320 | 2.900 |+| Predictive Maintenance | 1.115 | 1.508 | 1.960 | 2.450 |+| Energy Management & Carbon Neutrality | 0.299 | 0.675 | 0.980 | 1.420 |+| **Total Revenue** | **3.105** | **3.968** | **5.260** | **6.770** |+| YoY Growth | 26.6% | 27.8% | 32.5% | 28.7% |++### Profit Forecast++| Metric | 2024A | 2025E | 2026E | 2027E |+|---|---|---|---|---|+| Gross Margin | 41.2% | 44.2% | 46.5% | 48.0% |+| Net Margin | 5.0% | 9.1% | 12.5% | 14.8% |+| **Net Profit Attributable (RMB billion)** | **0.157** | **0.362** | **0.658** | **1.002** |+| Consensus Net Profit | 0.145 | 0.320 | 0.580 | 0.850 |+| Our vs Consensus | +8% | +13% | +13% | +18% |+| **EPS (RMB)** | **0.37** | **0.86** | **1.57** | **2.38** |++### Key Assumptions++- 2026 intelligent inspection business growth of 30%, benefiting from semiconductor industry recovery and new energy vehicle production capacity expansion+- Predictive maintenance business growth of 30%, with edge computing solutions contributing incremental revenue+- Energy management business growth of 45%, driven by CCER market restart and CBAM compliance demand+- Gross margin improving by 2 percentage points annually, mainly driven by increased subscription ratio+- R&D expense ratio maintaining at 18%-20%, selling expense ratio gradually declining with economies of scalediff --git a/tests/test_data/en/finance/financial_news_sample.md b/tests/test_data/en/finance/financial_news_sample.mdnew file mode 100644index 0000000..d638c68--- /dev/null+++ b/tests/test_data/en/finance/financial_news_sample.md@@ -0,0 +1,169 @@+# Financial Times Asia++# Weekly Macro Report: PBoC Cuts RRR by 50bps, Releasing Approximately RMB 1 Trillion in Liquidity++## January 13, 2025++**Author**: James Chen, Senior Macro Analyst, Beijing++---++## Key Events++### PBoC RRR Cut++The People's Bank of China announced on Friday that it will cut the reserve requirement ratio for financial institutions by 0.5 percentage points on January 15 (excluding financial institutions already subject to 5% RRR). After the cut, the weighted average RRR for financial institutions will be approximately 6.6%.++This is the first RRR cut by the PBoC since September 2024, and is expected to release approximately RMB 1 trillion in long-term liquidity.++The PBoC stated in its announcement that this RRR cut aims to "maintain reasonable and sufficient liquidity, promote a steady decline in comprehensive financing costs, and support the development of the real economy."++### Market Reactions++After the RRR cut announcement, financial markets responded positively:++- **A-Share Market**: The Shanghai Composite Index rose 0.62% to close at 3,256.39 points; the Shenzhen Component Index rose 0.83% to close at 10,978.35 points+- **Bond Market**: The 10-year Treasury yield dropped 4 basis points to 2.45%+- **RMB Exchange Rate**: The onshore RMB against USD rate remained basically flat at 7.24+- **Bank Stocks**: China Merchants Bank rose 3.2%, Bank of China rose 2.8%++---++## Policy Interpretation++### Background of RRR Cut++**1. Need for Economic Stabilization and Growth**++December 2024 CPI rose 0.1% year-over-year, PPI fell 2.7% year-over-year, with low inflation levels. GDP grew 4.8% for the full year 2024, below the 5% target set at the beginning of the year. The economy faces downward pressure and requires monetary policy support.++**2. Maintaining Reasonable Liquidity Abundance**++January is traditionally a peak season for credit issuance, plus increased cash demand before the Spring Festival, the banking system faces greater liquidity pressure. The RRR cut can inject liquidity into the banking system in advance to meet credit demand.++**3. Coordinating with Fiscal Policy**++The fiscal deficit ratio for 2025 is expected to increase to 4%, requiring monetary policy coordination. The RRR cut can reduce banks' liability costs and create conditions for banks to support the real economy.++### Impact of RRR Cut++**Impact on Banks**:+- Expected to reduce banks' liability costs by approximately RMB 150 billion+- Improve bank net interest margin by approximately 1-2 basis points+- Increase banks' loanable funds++**Impact on Real Economy**:+- Loan Prime Rate (LPR) is expected to further decline+- Corporate financing costs decline+- Real estate financing environment marginally improves++**Impact on Markets**:+- Short-term positive for stock market+- Bond market yield decline space opens up+- RMB exchange rate remains relatively stable++---++## Weekly Macro Data Review++### December 2024 Economic Data++| Indicator | Value | YoY/QoQ | Interpretation |+|---|---|---|---|+| CPI | +0.1% | QoQ flat | Low inflation level |+| PPI | -2.7% | QoQ flat | Industrial product prices under pressure |+| Exports (USD) | +2.3% | QoQ -4.7% | Strong external demand resilience |+| Imports (USD) | +0.4% | QoQ -6.3% | Domestic demand still weak |+| New RMB Loans | RMB 580B | YoY -RMB 120B | Weak credit demand |+| M2 | +7.3% | YoY -0.1 ppt | Steady monetary growth |++### Weekly Market Performance++| Market | Index/Price | Weekly Change |+|---|---|---|+| Shanghai Composite | 3,256.39 | +1.2% |+| Shenzhen Component | 10,978.35 | +1.8% |+| ChiNext Index | 2,052.68 | +2.3% |+| Hang Seng Index | 19,520.68 | +1.5% |+| 10Y Treasury Yield | 2.45% | -4bp |+| USD Index | 102.85 | +0.3% |+| Gold (USD/oz) | 2,048.5 | +0.8% |+| WTI Crude | $72.50/bbl | -2.1% |++---++## Institutional Views++### Brokerage Views++**CICC**: The PBoC RRR cut slightly exceeded expectations; LPR is expected to decline 5-10 basis points in Q1. Maintaining the forecast of 2 RRR cuts and 1 interest rate cut in 2025.++**Huatai Securities**: The RRR cut directly benefits bank stocks; continuing to favor high-quality bank stocks such as China Merchants Bank and Ningbo Bank.++**Guotai Junan**: The RRR cut benefits A-shares, especially financial and cyclical stocks. However, the economic fundamentals still need more policy support.++### Institution Views++**Goldman Sachs**: Expects China's 2025 GDP growth at 4.5%, with increased policy easing intensity.++**Morgan Stanley**: The RRR cut is a positive policy signal, but insufficient to completely offset downward pressure from the real estate sector.++---++## Policy Outlook++### Monetary Policy++**Short-term**:+- LPR expected to decline 5-10 basis points in Q1+- Structural monetary policy tools continue to play a role+- Refinancing scale may expand++**Full Year Outlook**:+- RRR cut 1-2 times (50 basis points each time)+- MLF rate decline once (approximately 10 basis points)+- LPR decline 1-2 times (total 10-20 basis points)++### Fiscal Policy++- Fiscal deficit ratio for 2025 expected to increase to 4%+- New special bond quota approximately RMB 4.5 trillion+- Ultra-long-term special Treasury bond issuance approximately RMB 1 trillion+- Acceleration of local government implicit debt replacement++---++## Risk Alerts++1. **Real Estate Risk**: Continuous decline in real estate sales and investment may trigger systemic risk+2. **Local Debt Risk**: Local government debt pressure is relatively large, requiring proper policy resolution+3. **External Uncertainty**: Federal Reserve monetary policy direction and geopolitical changes may affect domestic markets+4. **Inflation Risk**: If pork prices rise sharply, it may push up CPI++---++## Investment Recommendations++### Sector Allocation++Based on the RRR cut and policy benefits, the following sectors are recommended:++| Sector | Logic | Recommended Stocks |+|---|---|---|+| Banks | Declining liability costs, stabilizing net interest margin | China Merchants Bank, Ningbo Bank |+| Securities | Improved market liquidity, trading volume expected to recover | CITIC Securities, Huatai Securities |+| Real Estate | Marginally improved financing environment | Vanke A, Poly Development |+| Cyclicals | Improved economic expectations | Conch Cement, PetroChina |+| Tech Growth | Improved risk appetite | CATL, BYD |++### Risk Alerts++- The above views are for reference only and do not constitute investment advice+- Markets involve risks, and investment requires caution+- Please make investment decisions based on your own risk tolerance++---++*Author James Chen is a senior macro analyst at Financial Times Asia*++*Contact: [email protected]*diff --git a/tests/test_data/en/finance/ipo_prospectus_sample.md b/tests/test_data/en/finance/ipo_prospectus_sample.mdnew file mode 100644index 0000000..d978674--- /dev/null+++ b/tests/test_data/en/finance/ipo_prospectus_sample.md@@ -0,0 +1,101 @@+# Prospectus++# QuantumCloud Technology Co., Ltd.++## Initial Public Offering and Listing on the STAR Market++---++| Item | Content |+|---|---|+| Issuer | QuantumCloud Technology Co., Ltd. |+| Type of Shares Issued | RMB Ordinary Shares (A Shares) |+| Number of Shares Issued | 25,000,000 shares (excluding over-allotment option) |+| Issue Price | RMB 68.50 per share |+| Estimated Total Proceeds | RMB 1.713 billion |+| Estimated Net Proceeds | RMB 1.580 billion |+| Total Shares After Issuance | 100,000,000 shares |+| Listing Exchange | Shanghai Stock Exchange STAR Market |+| Sponsor (Lead Underwriter) | China International Capital Corporation Limited |+| Co-underwriters | Huatai Securities, CITIC Securities |+| Prospectus Signing Date | December 15, 2025 |++---++## Issuer's Statement++The Company's issuance application still needs to undergo corresponding review and registration procedures by the Shanghai Stock Exchange and the China Securities Regulatory Commission. This prospectus does not have legal effect for share issuance and is for pre-disclosure purposes only. Investors should make investment decisions based on the full text of the formally announced prospectus.++**Special Risk Warning**: The Company belongs to the artificial intelligence industry, with large R&D investment and long profitability cycle. The Company achieved net profit of RMB -32.5 million, -18.5 million, and RMB 21.8 million for 2023, 2024, and 2025 respectively, with cumulative deficit of RMB 29.2 million. The Company achieved profitability for the first time in 2025, and the sustainability of profitability remains uncertain.++---++## Section 1: Company Overview++### 1.1 Company Introduction++QuantumCloud Technology Co., Ltd. (hereinafter referred to as "QuantumCloud," "the Company" or "the Issuer") was established in 2018 as a high-tech enterprise specializing in quantum computing simulation and quantum security encryption technology. The Company's headquarters is located in Hefei High-tech Industrial Development Zone, with R&D centers in Beijing, Shanghai and Shenzhen.++The Company's main business includes:+1. **Quantum Computing Simulation Platform** (45% of revenue): Quantum circuit simulation software based on classical computers, helping research institutions and enterprises develop and verify quantum algorithms before quantum hardware matures+2. **Quantum Security Encryption Products** (38% of revenue): Post-quantum cryptography (PQC) solutions providing data security protection against quantum computing attacks for finance, government, military and other fields+3. **Quantum Computing Education and Consulting** (17% of revenue): Quantum computing training courses and technical consulting services for universities and enterprises++### 1.2 Company Development History++#### 2018 — Founding++- **March 2018**: Company founded by Co-founders Dr. Raymond Liu (CEO) and Dr. Grace Chen (CTO) in Hefei, based on their research at the Quantum Information Laboratory of University of Science and Technology of China. Dr. Liu previously worked on quantum computing research at D-Wave Systems in Canada+- **May 2018**: Received angel round financing of RMB 5 million, led by Hefei Science and Technology Innovation Investment Co., Ltd.+- **September 2018**: Core team expanded to 15 people; completed quantum circuit simulation engine v0.1 prototype development++#### 2019 — Technology Accumulation++- **April 2019**: Established first quantum computing laboratory in Hefei High-tech Zone, covering 800 square meters+- **August 2019**: Received Series A financing of RMB 30 million, led by CAS Innovation Ventures, with Hefei Industry Investment following+- **December 2019**: Released quantum circuit simulation platform "CloudQ Q-Sim" v1.0, supporting 20-qubit simulation, with leading performance among domestic similar products++#### 2020 — Product Commercialization++- **June 2020**: First commercial customer acquired, signed quantum security encryption POC contract with Bank of China Information Technology Department for RMB 2.8 million+- **November 2020**: Received Series B financing of RMB 120 million, led by Sequoia Capital China, with IDG Capital and Shenzhen Capital following, post-money valuation of RMB 800 million+- **December 2020**: Employee count increased to 52; annual operating revenue of RMB 18.5 million++#### 2021 — Scale Expansion++- **March 2021**: Established R&D center in Beijing Zhongguancun, focusing on productization of quantum security encryption technology+- **July 2021**: Obtained commercial cryptography product model certificate issued by the State Cryptography Administration, becoming one of the first domestic enterprises to obtain post-quantum cryptography product certification+- **October 2021**: Received Series B+ financing of RMB 250 million, led by SDIC Venture, valuation of RMB 1.8 billion+- **December 2021**: Number of contracted customers reached 85; annual operating revenue of RMB 56.8 million++#### 2022 — Business Breakthrough++- **May 2022**: Signed quantum secure communication upgrade project contract with State Grid Information and Communication Industry Group for RMB 85 million, the largest single order in the Company's history+- **September 2022**: "CloudQ Q-Sim" v3.0 released, supporting 50-qubit full-amplitude simulation, partnering with Alibaba Cloud to provide cloud-based quantum simulation services+- **November 2022**: Received Series C financing of RMB 380 million, led by Hillhouse Ventures, valuation of RMB 2.8 billion+- **December 2022**: Employee count increased to 185; annual operating revenue of RMB 235 million; net loss of RMB 18.5 million++#### 2023 — Industry Deepening++- **April 2023**: Established Financial Industry Solutions Center in Shanghai Zhangjiang, focusing on providing quantum security solutions for banking, securities and insurance industries+- **August 2023**: Signed strategic cooperation agreements with ICBC, CCB and China Merchants Bank to promote quantum security encryption upgrades in the financial industry+- **October 2023**: Quantum security VPN product obtained CCRC information security certification+- **December 2023**: Number of contracted customers reached 280; annual operating revenue of RMB 452 million; net loss of RMB 32.5 million++#### 2024 — Technology Milestone++- **March 2024**: "CloudQ Q-Sim" v5.0 released, supporting 100-qubit simulation and quantum error correction simulation functions, ranking among the top three globally in international quantum computing simulation benchmarks+- **August 2024**: Received Series D financing of RMB 600 million, led by National Manufacturing Transformation and Upgrade Fund, valuation of RMB 5.2 billion+- **December 2024**: Employee count increased to 420; annual operating revenue of RMB 680 million; net loss of RMB 18.5 million++#### 2025 — Approaching Profitability++- **March 2025**: Signed quantum security government cloud project contract with Shenzhen Municipal Government Data Administration Bureau for RMB 120 million+- **June 2025**: Started overseas business, signed quantum security cooperation MOUs with Singapore Telecom and Korea's KT Corporation+- **October 2025**: Number of contracted customers reached 520; annual operating revenue of RMB 920 million; net loss narrowed to RMB 4.2 million++#### Q3 2025 — First Profitability and IPO Preparation++- **September 2025**: Achieved profitability for the first time in a single quarter (Q3 2025 net profit of RMB 18.5 million)+- **November 2025**: Completed joint-stock reform, converted to a joint-stock limited company+- **December 2025**: Submitted IPO application to Shanghai Stock Exchangediff --git a/tests/test_data/en/finance/supply_chain_analysis_sample.md b/tests/test_data/en/finance/supply_chain_analysis_sample.mdnew file mode 100644index 0000000..ceeb84b--- /dev/null+++ b/tests/test_data/en/finance/supply_chain_analysis_sample.md@@ -0,0 +1,100 @@+# New Energy Vehicle Supply Chain Risk Assessment Report++**Prepared by**: Deloitte Consulting (Shanghai) Co., Ltd.+**Client**: Auric New Energy Vehicle Technology Co., Ltd.+**Report Date**: November 2025+**Classification**: Confidential — For Client Internal Use Only++---++## Executive Summary++This report provides a comprehensive risk assessment of the core component supply chain for Auric New Energy Vehicle Technology Co., Ltd. (hereinafter referred to as "Auric" or "the Company"). The assessment scope covers 38 Tier-1 suppliers, 96 Tier-2 suppliers, and key raw material supply sources, involving 12 countries and regions. The report identifies 6 high-risk supply chain nodes and provides mitigation recommendations for each risk.++---++## Supply Chain Architecture++### 2.1 Core Component Tier-1 Suppliers++#### Power Battery++| Supplier | Location | Supply Product | Annual Procurement (RMB billion) | Supply Share | Risk Level |+|---|---|---|---|---|---|+| **CATL** | Ningde, Fujian | Ternary lithium/P lithium battery packs | 4.85 | 58% | Medium |+| **BYD FinDreams Battery** | Chongqing | Blade battery packs | 2.20 | 26% | Low |+| **CALB** | Changzhou, Jiangsu | Ternary lithium (backup supplier) | 1.35 | 16% | Medium |++#### Electric Drive System++| Supplier | Location | Supply Product | Annual Procurement (RMB billion) | Supply Share | Risk Level |+|---|---|---|---|---|---|+| **Inovance** | Shenzhen, Guangdong | Permanent magnet synchronous motors, motor controllers | 1.82 | 52% | Medium |+| **Nidec** | Kyoto, Japan | Drive motor assembly (E-Axle) | 1.25 | 36% | High |+| **JJE** | Beijing | Hybrid electric motors | 0.42 | 12% | Low |++#### Intelligent Driving Chips++| Supplier | Location | Supply Product | Annual Procurement (RMB billion) | Supply Share | Risk Level |+|---|---|---|---|---|---|+| **NVIDIA** | Santa Clara, CA | Orin X intelligent driving SoC | 0.86 | 65% | High |+| **Horizon Robotics** | Beijing | Journey 5 chip | 0.32 | 24% | Medium |+| **Black Sesame** | Shanghai | A1000 Pro chip | 0.15 | 11% | Medium |++#### Body and Chassis++| Supplier | Location | Supply Product | Annual Procurement (RMB billion) | Risk Level |+|---|---|---|---|---|+| **Tuopu Group** | Ningbo, Zhejiang | Chassis suspension systems, lightweight components | 0.68 | Low |+| **Zhongding** | Ningguo, Anhui | Rubber seals, fluid piping | 0.35 | Low |+| **Fuyao Glass** | Fuqing, Fujian | Automotive glass, HUD抬头显示玻璃 | 0.42 | Low |+| **Joyson Electronic** | Ningbo, Zhejiang | Safety systems (airbags, steering wheels) | 0.51 | Medium |++#### Cabin and Connected Vehicle++| Supplier | Location | Supply Product | Annual Procurement (RMB billion) | Risk Level |+|---|---|---|---|---|+| **Qualcomm** | San Diego, CA | 8295 cabin chip | 0.48 | High |+| **Desay SV** | Huizhou, Guangdong | Intelligent cockpit domain controller | 0.56 | Medium |+| **BOE** | Beijing | Vehicle display screens | 0.32 | Low |++---++### 2.2 Tier-2 Supplier Key Dependencies++**Lithium Battery Key Raw Materials**++- **Tianqi Lithium** (Shehong, Sichuan) → Supplies battery-grade lithium carbonate to CATL. Tianqi controls approximately 10% of global lithium ore resources, and its Greenbushes mine in Australia is the world's largest hard rock lithium mine. Lithium carbonate price has fallen from RMB 600,000/ton at the 2022 peak to approximately RMB 80,000/ton, but supply-side concentration risk remains.++- **Huayou Cobalt** (Tongxiang, Zhejiang) → Supplies battery-grade cobalt sulfate and precursors to CATL and CALB. Cobalt resources are 70% from DRC, with geopolitical risk and ESG compliance risk.++- **GEM** (Jingmen, Hubei) → Battery recycling and precursor materials. The company is China's largest power battery recycling enterprise with recycling capacity of 300,000 tons/year, helping reduce dependence on upstream mineral resources.++**Semiconductor Manufacturing Chain**++- **TSMC** (Hsinchu, Taiwan) → Manufactures intelligent driving/cabin chips for NVIDIA and Qualcomm on behalf. NVIDIA Orin X chip uses Samsung 8nm process, but next-generation Thor chip will move to TSMC 5nm. Qualcomm 8295 uses TSMC 5nm process. TSMC is an irreplaceable key node in the intelligent driving chip supply chain.++- **ASE** (Kaohsiung, Taiwan) → Provides advanced packaging services for NVIDIA and Qualcomm. Packaging is currently one of the most capacity-constrained segments in the chip supply chain.++**Rare Earth Permanent Magnet Materials**++- **Northern Rare Earth** (Baotou, Inner Mongolia) → Supplies NdFeB permanent magnet materials to Inovance and Nidec for manufacturing permanent magnets in drive motors. China controls approximately 60% of global rare earth minerals and 90% of rare earth processing capacity. Since 2025, rare earth export control policies have tightened.++---++## Risk Assessment++### 3.1 Geopolitical Risk — Extremely High++**US-China Tech Competition and Chip Controls**++The largest geopolitical risk in Auric's supply chain comes from US-China tech decoupling. The Company has high dependence on US-based chips:++- NVIDIA Orin X intelligent driving chip: accounts for 100% chip supply for the Company's advanced intelligent driving solutions, with annual procurement of RMB 86 million+- Qualcomm 8295 cabin chip: accounts for 80% chip supply for the Company's mid-to-high-end cabin solutions, with annual procurement of RMB 48 million++**Risk Assessment**: The US Department of Commerce has upgraded chip export controls to China multiple times in 2022 and 2023. While current controls primarily target advanced AI training chips, there is no guarantee that automotive high-computing chips will not be included in the control list in the future. Once NVIDIA and Qualcomm chips are prohibited from exporting to China, the Company's advanced intelligent driving and cabin system production will face production stoppage risk after existing inventory is exhausted (approximately 4-6 months).++**Mitigation Measures**:+1. Accelerate adaptation and validation work for Horizon Robotics Journey 6 chip, planned to complete full model adaptation by Q3 2026 (expected investment of RMB 120 million)+2. Initiate cooperation evaluation with Huawei MDC intelligent driving platformdiff --git a/tests/test_data/en/general/arbitrary_tech_news.md b/tests/test_data/en/general/arbitrary_tech_news.mdnew file mode 100644index 0000000..9b89ffd--- /dev/null+++ b/tests/test_data/en/general/arbitrary_tech_news.md@@ -0,0 +1,96 @@+# Galactic Travels Announces Shenzhou-50 Maiden Flight Success, Mars Travel Ticket Price Drops to 2 Million Yuan++**Publication Date**: June 18, 2090+**Author**: Space Reporter Starry Walker++---++## Core Announcement++Today, **Galactic Travels**, the world's leading commercial space travel company, held a grand press conference at the **Wenchang Space Launch Center**, officially announcing the successful maiden flight of its next-generation manned spacecraft — **Shenzhou-50**!++Galactic Travels Founder and CEO **Richard Xinghe**激动地表示:"Shenzhou-50's successful maiden flight marks a new era in human space travel. Our goal is to make space travel accessible to every ordinary person!"++---++## Shenzhou-50 Highlights++### 1. Revolutionary Propulsion System++Shenzhou-50 is equipped with Galactic Travels' self-developed **"Galactic Engine" nuclear propulsion system**, a major breakthrough in human aerospace history:+- Earth to Mars travel time reduced from 6 months to **30 days**+- Propulsion efficiency improved by 500%+- Maximum speed can reach 2% of the speed of light++### 2. Comfortable Space Cabin Experience++Shenzhou-50 can accommodate 20 passengers, providing a luxurious and comfortable space travel experience:+- Each cabin equipped with panoramic skylights for viewing magnificent cosmic scenery+- Zero-gravity fitness center and recreational area+- Artificial gravity cabin to reduce space discomfort+- Michelin three-star space restaurant++### 3. Safety Assurance System++Addressing special risks of space travel, Shenzhou-50 provides multiple safety assurances:+- Triple-redundant life support system+- Micro black hole protective shield+- Emergency return capsule, capable of returning to Earth within 24 hours+- ISO Space Safety Certification compliant++---++## Space Team++Shenzhou-50's R&D was led by Galactic Travels CTO **Kevin Yuzhou**, with core team members including:+- **Sarah Xingchen**: Chief Designer, previously participated in International Space Station construction+- **Michael Xingyun**: Propulsion System Lead, nuclear power expert+- **David Yinghe**: Mission Planning Director, with 20 years of aerospace mission experience+- **Emily Xingji**: Safety Systems Architect++The entire R&D team consists of over 5,000 people, completing Shenzhou-50's R&D in 10 years.++---++## Strategic Partnerships++During the press conference, Galactic Travels simultaneously announced strategic partnerships with multiple institutions:+- **International Aerospace Agency (IAA)**: Jointly develop deep space exploration technology+- **Mars Base Construction Company**: Build the first commercial base on Mars+- **Space Hotel Group**: Jointly operate orbital space station hotels+- **Interstellar Tourism Alliance**: Establish global space travel standards++---++## Development Timeline++Galactic Travels was founded in 2050, headquartered in Shanghai:+- October 2050: Secured angel round financing of 1 billion yuan+- May 2055: Completed Series A financing of 5 billion yuan+- August 2060: Shenzhou-30 maiden flight successful, achieving suborbital travel+- March 2070: Series B financing of 20 billion yuan, valuation reaching 100 billion yuan+- December 2080: Shenzhou-40 maiden flight successful, achieving lunar commercial travel+- June 2090: Shenzhou-50 maiden flight successful, opening Mars travel era++---++## Pricing and Booking++Shenzhou-50 Mars travel will officially accept bookings on October 1, 2090:+- **Economy Class**: 2 million yuan/person+- **Business Class**: 5 million yuan/person+- **Luxury Class**: 20 million yuan/person++Early booking users can enjoy a 50% discount!++---++## Market Outlook++Industry analysts believe that Shenzhou-50's launch will completely change human travel methods. It is estimated that by 2100, the global space travel market will exceed 10 trillion yuan.++Galactic Travels stated that it will continue to invest in R&D, planning to launch **Shenzhou-60** in 2095 to achieve Saturn travel, and achieve commercial travel to any planet in the solar system by 2100.++---++"The universe is so vast, we all want to see it!" — Galactic Travels Founder Richard Xinghediff --git a/tests/test_data/en/general/biography_scientist.md b/tests/test_data/en/general/biography_scientist.mdnew file mode 100644index 0000000..043c5b6--- /dev/null+++ b/tests/test_data/en/general/biography_scientist.md@@ -0,0 +1,80 @@+# Zu Chongzhi: A Great Scientist of Ancient China++**Author**: Historical Biography by Chen Jingyi+**Publication Year**: 2026++---++## Preface++Zu Chongzhi was an outstanding mathematician, astronomer, and inventor during the Northern and Southern Dynasties of China. He achieved remarkable accomplishments in multiple fields including mathematics, astronomical calendar, and mechanical manufacturing, standing as a monument in the history of ancient Chinese science.++This book aims to document the extraordinary life of Zu Chongzhi, showcasing his steadfast pursuit of science and great contributions to Chinese civilization.++---++## Chapter 1: Early Years (429-450)++### 1.1 Birth and Family++In 429 (the sixth year of the Yuanjia era under Emperor Wen of Liu Song), Zu Chongzhi was born in **Fanyang Commandery Qiuxian** (now Laishui County, Hebei Province) into an official family.++His great-grandfather **Zu Taizhi** served as Imperial Secretary and Guanglu Doctor during the Eastern Jin Dynasty. His grandfather **Zu Chang** held the position of Grand Master of Architecture (an official overseeing civil engineering) during the Liu Song Dynasty. His father **Zu Shuo Zhi** served as a court official. In this family environment, Zu Chongzhi received excellent education from an early age and developed a strong interest in science and technology.++### 1.2 Youth and Love of Learning++Zu Chongzhi was studious from childhood, reading extensively, with a particular fondness for books on mathematics, astronomy, and mechanical manufacturing.++Historical records describe Zu Chongzhi as "having extraordinary intelligence and thinking ability, studying ancient classics from youth."++### 1.3 Entering Officialdom++Around 450, Zu Chongzhi entered official service, serving as an official historian in Southern Xuzhou (now Zhenjiang, Jiangsu), later becoming a military officer in the duke's residence. Although in officialdom, he never abandoned his pursuit of scientific research.++---++## Chapter 2: Mathematical Achievements (450-470)++### 2.1 Calculation of Pi++Zu Chongzhi's most outstanding contribution to mathematics was the precise calculation of pi (π).++After long and arduous research, Zu Chongzhi determined:+- **Ying Shu (Upper Limit)**: 3.1415927+- **Nü Shu (Lower Limit)**: 3.1415926++That is, the value of pi lies between **3.1415926 and 3.1415927**, accurate to the 7th decimal place. This achievement preceded Western mathematicians by **over a thousand years**!++For convenience in calculation, Zu Chongzhi also provided two fractional approximations:+- **Yuelü (Approximate Ratio)**: 22/7 (≈3.1429)+- **Milü (Precise Ratio)**: 355/113 (≈3.1415929)++Among them, the precise ratio 355/113 is the best approximate fraction with numerator and denominator under 1000, also leading the world by over a thousand years.++### 2.2 "Zhuishu"++Zu Chongzhi compiled his mathematical research into a work called **"Zhuishu"** (Mathematical Treatise).++"Zhuishu" was profound in content, known as a "classic of mathematical studies," and later became one of the textbooks for the mathematics department of the Imperial Academy during the Tang Dynasty. Unfortunately, this precious mathematical work was lost during the Northern Song Dynasty.++---++## Chapter 3: Astronomical Calendar (450-464)++### 3.1 Discovering Calendar Problems++While researching astronomical calendars, Zu Chongzhi found numerous errors in the prevailing **"Yuanjia Calendar"**.++The Yuanjia Calendar was formulated by astronomer He Chengtian. Although more precise than previous calendars, it still had deficiencies. After years of observation and research, Zu Chongzhi pointed out multiple flaws in the Yuanjia Calendar.++### 3.2 Creating the "Daming Calendar"++In 462 (the sixth year of the Daming era under Emperor Xiaowu of Liu Song), Zu Chongzhi compiled a new calendar — the **"Daming Calendar"**.++The main innovations of the Daming Calendar include:+1. **Introduction of Precession**: A major breakthrough in Chinese calendar history, distinguishing between tropical year and sidereal year+2. **Improved Lunisolar Cycle**: Changed from the old calendar's 19-year-7-intercalation to 391-year-144-intercalation, more precise+3. **Accurate Determination of Nodal Month**: Calculated the nodal month as 27.21223 days, differing minimally from the modern measurement of 27.21222 days+4. **Precise Determination of Five Planets' Cycles**: Conducted precise measurements of the orbital periods of the five planets: Mercury, Venus, Mars, Jupiter, and Saturn++### 3.3 Calendar Debatediff --git a/tests/test_data/en/general/cross_reference_coffee.md b/tests/test_data/en/general/cross_reference_coffee.mdnew file mode 100644index 0000000..42fd79d--- /dev/null+++ b/tests/test_data/en/general/cross_reference_coffee.md@@ -0,0 +1,61 @@+# Coffee++**Beverage Type**: Caffeinated Drink+**Main Ingredients**: Coffee Beans, Water+**Common Consumption Scenarios**: Breakfast, Socializing, Work, Relaxation++---++## Overview++**Coffee** is a beverage made primarily from roasted coffee beans, and is one of the most common hot drinks in modern urban life. It can be consumed alone or combined with milk, syrup, cream, and other ingredients to create beverages with distinctly different flavors. Because coffee has stimulating effects, people typically view it as a common choice for morning workdays, afternoon breaks, and social gatherings.++In daily experience, coffee is frequently discussed alongside **tea**, as both are widely popular beverages containing stimulating ingredients, but differ significantly in flavor, preparation methods, and consumption culture. To understand the stimulating effects of coffee from an ingredient perspective, one typically needs to further consult "**Caffeine**."++---++## Ingredients and Names++The ingredients for coffee come from the seeds of the coffee tree, commonly known as coffee beans. After harvesting, coffee beans undergo processing, drying, roasting, and grinding before entering the brewing stage. The degree of roasting significantly affects bitterness, acidity, and aroma. Therefore, many articles introducing coffee mention "**roasting**" and "**grinding**."++In consumer scenarios, people often don't directly接触生豆处理过程 (come into contact with raw bean processing), but rather encounter coffee through store beverages, drip bags, instant products, or home brewing equipment. Therefore, popular entries often focus less on agricultural production and more on finished product flavors, brewing methods, and drinking culture.++---++## Historical Spread++Coffee was originally not a global daily beverage; its spread underwent a lengthy historical process. With the expansion of trade networks, the formation of colonial plantation systems, and the development of urban public life, coffee gradually transformed from a regional beverage into a cross-cultural consumer product. In modern times, coffee is not merely a drink but also part of commercial spaces, reading, writing, and social activities.++In European urban history and modern public culture studies, coffee is frequently discussed alongside "**cafés**" because many intellectual exchanges, literary activities, and civic gatherings were related to café spaces. Regarding this spatial form and its social functions, see "café."++---++## Preparation Methods++Coffee flavor is influenced not only by bean variety but also by brewing methods. Common methods in home environments include drip brewing, immersion, French press, and capsule machine brewing; in professional stores, **espresso** based on high-pressure extraction is commonly seen. Espresso can be consumed alone or used as a base for other beverages, which is why many coffee knowledge entries list "espresso" as a related entry.++Adding steamed milk to espresso typically produces a **latte**; varying proportions of milk and milk foam may create a **cappuccino**. If introducing coffee from the perspective of home preparation and flavor control, text often shifts toward "**pour-over coffee**" because it better demonstrates the impact of water temperature, grind size, and pouring methods on flavor.++Therefore, in beverage classification, the general "coffee" entry often maintains stable cross-reference relationships with entries like "espresso," "latte," "cappuccino," and "pour-over coffee."++---++## Flavor and Components++Coffee flavor is typically composed of acidity, bitterness, sweetness, body, and aroma layers. Different origins, roasting methods, and extraction techniques can present the same beans with noticeably different tastes. For ordinary consumers, the most easily perceived component influence is still the **caffeine** content. Regarding coffee's effects on alertness, sleep, and heart rate, discussions typically do not expand within the "coffee" entry but refer to "caffeine."++In beverage comparisons, coffee is also frequently contrasted with **cocoa** or tea. Compared to cocoa, coffee typically emphasizes roasted aroma and bitterness; compared to tea, coffee in modern consumption contexts is more often associated with takeaway, stores, and office scenarios. For such comparative reading, see "tea" and "cocoa."++---++## Daily Culture++In many countries and regions, coffee has long exceeded the beverage itself to become part of daily rhythm. People buy coffee during commutes, briefly stop by during meeting breaks, and treat weekend store visits and tasting new beans as leisure activities. With the proliferation of chain brands, independent stores, and home brewing equipment, coffee consumption has formed very rich layers. This culture also maintains a stable connection between "coffee" and "café." For many readers, understanding coffee is not just understanding a liquid, but understanding an urban experience unfolding around space, socializing, and consumption patterns.++---++## Health and Controversy++Regarding whether coffee is healthy, there have long been differing viewpoints. Proponents typically emphasize the benefits of moderate consumption for alertness and maintaining concentration; critics worry about high-frequency intake affecting sleep, anxiety levels, or cardiovascular status. Since most of these discussions are directly related to caffeine intake, relevant materials often further direct discussions toward "caffeine."++Additionally, differences between instant coffee, chain store beverages, and specialty coffee are frequently discussed by consumers and media. The focus of controversy usually lies not in "whether coffee is coffee," but in flavor standards, added ingredients, pricing systems, and consumption experiences.diff --git a/tests/test_data/en/general/encyclopedia_machine_learning.md b/tests/test_data/en/general/encyclopedia_machine_learning.mdnew file mode 100644index 0000000..c5d0d94--- /dev/null+++ b/tests/test_data/en/general/encyclopedia_machine_learning.md@@ -0,0 +1,290 @@+# Machine Learning++**Disciplinary Classification**: Artificial Intelligence+**Creation Date**: February 22, 2026+**Last Updated**: February 22, 2026++---++## Overview++**Machine Learning (ML)** is an important branch of **Artificial Intelligence (AI)** dedicated to researching how computer systems can automatically improve through experience.++The core idea of machine learning is: not relying on explicit hard-coded rules, but learning patterns and rules from data through algorithms.++---++## Definition++Machine learning is a multidisciplinary field involving **probability theory**, **statistics**, **approximation theory**, **convex analysis**, **computational complexity theory**, and other disciplines.++It is the core of artificial intelligence and the fundamental approach to giving computers intelligence. Machine learning algorithms analyze large amounts of data, automatically discover patterns, and use these patterns to make predictions or decisions on unknown data.++---++## Classification System++### 1. Supervised Learning++Supervised learning uses **labeled training data** for learning. Algorithms learn mapping relationships from input features and corresponding output labels.++**Common Algorithms**:+- Linear Regression+- Logistic Regression+- Decision Tree+- Random Forest+- Support Vector Machine (SVM)+- Neural Networks++**Application Scenarios**:+- House price prediction+- Spam email classification+- Image recognition+- Medical diagnosis++### 2. Unsupervised Learning++Unsupervised learning uses **unlabeled data**, where algorithms need to discover structures and patterns in the data on their own.++**Common Algorithms**:+- Clustering: K-Means, Hierarchical Clustering, DBSCAN+- Dimensionality Reduction: PCA, t-SNE, Autoencoders+- Association Rule Mining: Apriori, FP-Growth++**Application Scenarios**:+- Customer segmentation+- Anomaly detection+- Recommendation systems+- Topic modeling++### 3. Reinforcement Learning (RL)++Reinforcement learning learns through **interaction between an Agent and an Environment**. The agent obtains rewards through trial and error, with the goal of maximizing long-term cumulative rewards.++**Core Concepts**:+- State+- Action+- Policy+- Reward+- Value Function++**Common Algorithms**:+- Q-Learning+- Policy Gradient+- Actor-Critic+- DQN (Deep Q-Network)+- PPO (Proximal Policy Optimization)++**Application Scenarios**:+- Game AI (such as AlphaGo)+- Robot control+- Autonomous driving+- Resource scheduling++### 4. Semi-Supervised Learning++Combines small amounts of labeled data with large amounts of unlabeled data for learning.++### 5. Self-Supervised Learning++Constructs supervisory signals from the data itself without requiring manual labeling.++---++## Key Concepts++### 1. Feature++Features are measurable attributes or variables that describe data. In machine learning, features are the representation of raw data input into the model.++**Examples**:+- In house price prediction: area, number of rooms, location, year built+- In spam classification: word frequency, sender address, email length+- In image recognition: pixel values, edges, textures++### 2. Model++A model is a mathematical representation that learns patterns from data. It can be linear models, tree-based models, neural networks, etc.++**Model Training**: The process of adjusting model parameters to minimize the difference between predictions and actual values.++**Model Evaluation**: Using metrics such as accuracy, precision, recall, F1 score to assess model performance.++### 3. Overfitting and Underfitting++- **Overfitting**: Model performs well on training data but poorly on new, unseen data. The model learns noise in the training data.+- **Underfitting**: Model performs poorly on both training data and new data. The model is too simple to capture data patterns.++### 4. Training Set, Validation Set, and Test Set++- **Training Set**: Data used for model training+- **Validation Set**: Data used for hyperparameter tuning and model selection+- **Test Set**: Data used to evaluate final model performance++---++## Common Algorithms++### Regression Algorithms++| Algorithm | Applicable Scenarios | Advantages | Disadvantages |+|---|---|---|---|+| Linear Regression | Continuous value prediction | Simple, interpretable | Only captures linear relationships |+| Ridge Regression | High-dimensional data | Handles multicollinearity | Feature selection not performed |+| Lasso Regression | High-dimensional data | Performs feature selection | May over-regularize |++### Classification Algorithms++| Algorithm | Applicable Scenarios | Advantages | Disadvantages |+|---|---|---|---|+| Logistic Regression | Binary classification | Simple, interpretable | Only captures linear boundaries |+| Decision Tree | Classification and regression | Easy to interpret | Prone to overfitting |+| Random Forest | Complex classification | High accuracy, handles missing values | Model is a black box |+| SVM | High-dimensional classification | Effective in high-dimensional spaces | Sensitive to parameter selection |+| Neural Networks | Complex pattern recognition | Can learn complex patterns | Requires large amounts of data |++### Clustering Algorithms++| Algorithm | Applicable Scenarios | Advantages | Disadvantages |+|---|---|---|---|+| K-Means | Spherical clusters | Fast, scalable | Requires specifying K value |+| DBSCAN | Arbitrary-shaped clusters | Automatically determines cluster count | Sensitive to parameter selection |+| Hierarchical | Small to medium datasets | Creates dendrogram showing hierarchy | Computationally intensive |++---++## Application Fields++### Computer Vision++- Image classification (e.g., ResNet, VGG)+- Object detection (e.g., YOLO, Faster R-CNN)+- Image segmentation (e.g., U-Net, Mask R-CNN)+- Face recognition++### Natural Language Processing++- Text classification+- Named entity recognition+- Machine translation+- Question answering systems+- Text generation++### Speech Recognition++- Speech-to-text+- Speaker recognition+- Voice assistants++### Recommendation Systems++- Collaborative filtering+- Content-based recommendation+- Hybrid recommendation systems++### Autonomous Driving++- Environmental perception+- Path planning+- Decision control++### Healthcare++- Medical image analysis+- Drug discovery+- Disease prediction+- Personalized treatment++---++## Development Trends++### Deep Learning Dominance++Since 2012, deep learning has dominated many machine learning benchmarks, especially in computer vision and natural language processing.++**Key Milestones**:+- 2012: AlexNet wins ImageNet competition+- 2014: GAN (Generative Adversarial Networks) proposed+- 2015: ResNet introduces residual connections+- 2017: Transformer architecture proposed+- 2020: GPT-3 demonstrates few-shot learning capabilities++### Large Language Models++Since 2020, large language models (LLMs) have developed rapidly:+- GPT series (OpenAI)+- BERT series (Google)+- Claude (Anthropic)+- LLaMA (Meta)++These models demonstrate emergent capabilities in tasks such as text generation, translation, summarization, and question answering.++### AI for Science++Machine learning is increasingly applied to scientific research:+- Protein structure prediction (AlphaFold)+- Materials discovery+- Climate modeling+- Drug discovery++### Edge AI++As hardware improves and model compression techniques advance, AI models are increasingly deployed on edge devices:+- Mobile phones+- IoT devices+- Embedded systems++---++## Ethical Considerations++### Bias and Fairness++ML models may inherit and amplify biases present in training data, leading to unfair decisions.++### Privacy Protection++ML models may inadvertently memorize sensitive information from training data.++### Interpretability++Complex ML models (especially deep learning) are often "black boxes," making their decisions difficult to interpret.++### Security++ML models face various security threats:+- Adversarial attacks+- Data poisoning+- Model inversion++---++## Learning Resources++### Recommended Books++1. "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" — Aurélien Géron+2. "Pattern Recognition and Machine Learning" — Christopher Bishop+3. "Deep Learning" — Ian Goodfellow, Yoshua Bengio, Aaron Courville++### Online Courses++1. Andrew Ng's Machine Learning Course (Stanford)+2. fast.ai Practical Deep Learning for Coders+3. Coursera Deep Learning Specialization++### Open Source Frameworks++- **TensorFlow**: Google-developed deep learning framework+- **PyTorch**: Facebook-developed deep learning framework+- **Scikit-learn**: Classic machine learning library+- **JAX**: Google's high-performance ML library++---++## Summary++Machine learning, as a core technology of artificial intelligence, is profoundly changing various aspects of society. With continuous algorithm innovation, expanding application scenarios, and increasing computational power, machine learning will continue to drive technological progress and social development.++Understanding the basic concepts, common algorithms, and application fields of machine learning is essential for professionals across disciplines.diff --git a/tests/test_data/en/general/regulation_company_policy.md b/tests/test_data/en/general/regulation_company_policy.mdnew file mode 100644index 0000000..9572a02--- /dev/null+++ b/tests/test_data/en/general/regulation_company_policy.md@@ -0,0 +1,79 @@+# Universe's Top Slackers Inc. Employee Attendance and Performance Management Policy++**Document Number**: MY-HR-2090-001+**Version**: V3.0+**Effective Date**: January 1, 2090+**Department**: Human Resources Department+**Approver**: King of Slackers, Wang Dacui++---++## Chapter 1: General Provisions++### 1.1 Purpose++To standardize employee attendance management, establish a scientific and reasonable performance evaluation system, protect the legitimate rights and interests of the Company and employees, and efficiently complete work while slackingly, this policy is formulated in accordance with the "Galactic Labor Law," "Universal Labor Contract Law," and related interstellar laws and regulations, combined with the Company's actual circumstances.++### 1.2 Scope of Application++This policy applies to all regular employees, probationary employees, and alien interns of Universe's Top Slackers Inc. (hereinafter referred to as "the Company").++### 1.3 Basic Principles++1. Balance between slacking and work principle+2. Principle of combining incentives with constraints+3. Principle of linking performance with slacking hours+4. Continuous slacking continuous improvement principle++---++## Chapter 2: Attendance Management Regulations++### 2.1 Working Hours++#### 2.1.1 Standard Working Hours++- **Workdays**: Monday to Friday (can apply to work from home slacking on any two days per week)+- **Working Hours**: 10:00 AM - 12:00 PM (slacking time), 2:00 PM - 5:00 PM (working time)+- **Lunch Break**: 12:00 PM - 2:00 PM (mandatory slacking time)+- **Weekly Working Hours**: Not exceeding 25 hours (remaining time can be freely arranged for slacking)++#### 2.1.2 Flexible Working Hours++The Company implements a **super flexible working system**, allowing employees to flexibly arrange work and slacking within the following ranges:+- Earliest arrival: 9:00 AM+- Latest arrival: 11:00 AM+- Earliest departure: 4:00 PM+- Latest departure: 6:00 PM++As long as 5 hours of work are completed daily, slacking time is unlimited.++#### 2.1.3 Special Positions++For positions requiring 24-hour duty such as space exploration and alien communication, a **shift system** is implemented. Specific scheduling is arranged by department heads, but for every 4 consecutive working hours, 1 hour of slacking is mandatory.++### 2.2 Clock-in Management++#### 2.2.1 Clock-in Methods++Employees must clock in through the Company's interstellar attendance system:+- **Brainwave Clock-in**: Direct clock-in using brain-computer interface+- **Holographic Projection Clock-in**: Clock-in through holographic image+- **Quantum Entanglement Clock-in**: Clock-in from anywhere in the universe++#### 2.2.2 Clock-in Requirements++- Clock-in required twice daily: arrival and departure+- Arrival clock-in should be completed within 30 minutes after arrival+- Departure clock-in should be completed within 30 minutes before or after leaving+- When unable to clock in due to official business (including travel to other planets), an "Outbound Request" should be submitted in advance through the OA system++#### 2.2.3 Forgetting to Clock In++- Forgetting to clock in is allowed 5 times per month+- When forgetting to clock in, a "Supplementary Clock-in Request" must be submitted through the OA system within 48 hours, effective after department head approval+- For the 6th and subsequent times exceeding 5, each instance is treated as 0.5 days of absence (but slacking time can offset it)++### 2.3 Tardiness, Early Leave and Absence++#### 2.3.1 Tardinessdiff --git a/tests/test_data/en/general/social_network_entrepreneur_story.md b/tests/test_data/en/general/social_network_entrepreneur_story.mdnew file mode 100644index 0000000..fc6f0b8--- /dev/null+++ b/tests/test_data/en/general/social_network_entrepreneur_story.md@@ -0,0 +1,37 @@+# She Brought Lab Methods to the Hospital: Lin Zhiyao and CloudSail AI's Five Years++**Source**: Urban Business Figures+**Column**: Entrepreneur Observer+**Publication Date**: March 2026++---++It is difficult to summarize **Lin Zhiyao** with a single identity. To investors, she is the Founder and CEO of **CloudSail AI**; to hospital clients, she is someone who can explain models clearly and follow through on implementation details; and in the eyes of teachers and classmates from the **Tsinghua University Artificial Intelligence Lab**, she is still the student who worked quickly and often sat in the front row during meetings.++Lin Zhiyao was born in Hangzhou into a typical intellectual family. Her father **Lin Guosheng** was an architectural designer, and her mother **Xu Manqing** worked as an editor at a publishing house. The family often gathered around the dinner table discussing industry news and manuscript topics. She was six years older than her younger sister **Lin Yuwei**, who later became a documentary filmmaker. Both sisters are busy with work, but during holidays Lin Yuwei typically handles photography while Lin Zhiyao coordinates family dinner timing. During Lin Zhiyao's first two years of entrepreneurship, the earliest photo hanging on the office wall was a team photo taken by Lin Yuwei.++During undergraduate studies, Lin Zhiyao majored in Computer Science at Tsinghua University. What truly influenced her career direction was the period after entering the Tsinghua University Artificial Intelligence Lab for graduate studies. Lab director **Prof. Shen Bowen** was one of the earlier domestic scholars working on medical intelligent decision research. When Lin Zhiyao first met Prof. Shen, he had just returned from a hospital collaboration site, rain still on his shoes. Later, she mentioned on many occasions that her initial understanding of "algorithms are not algorithms in papers, but algorithms in scenarios" came from Prof. Shen that day.++In the lab, Lin Zhiyao was close to PhD senior **He Meng**, who led medical data projects. Lin Zhiyao often went with her to hospitals to organize annotation processes and confirm field specifications with doctors. When both recalled that period later, they said the hardest part wasn't tuning model parameters, but repeatedly meeting with clinical teams over disease stratification standards. For a while, He Meng would almost daily call Lin Zhiyao back to the lab after dinner to supplement experimental records, while Lin Zhiyao was responsible for redrawing the charts for the next day's discussion.++It was also at Tsinghua that Lin Zhiyao met **Zhou Kai**. They were originally undergraduate classmates, but truly became close after a cross-department machine learning competition. Zhou Kai excelled at system architecture while Lin Zhiyao was better at breaking down problems clearly. Around graduation, they participated in almost every project they could form teams for. Later, Zhou Kai continued joint training research at the university and often sat in on group meetings at Prof. Shen's team. Lab members习惯把他算作"半个自己人" (treated him as "half one of us"), and Lin Zhiyao still calls Zhou Kai "old classmate" in public speeches.++In 2022, Lin Zhiyao and Zhou Kai co-founded CloudSail AI. When the company started, there were only about ten people. Lin Zhiyao was responsible for external financing, industry partnerships, and overall product direction, while Zhou Kai led the engineering team building the underlying system. Early employees recalled that the company's largest meeting room was actually a temporarily converted storage room, where Lin Zhiyao often held video calls with clients while having Zhou Kai redraw deployment architecture on the whiteboard. When the company began having clear divisions of labor, Zhou Kai's team still reported directly to Lin Zhiyao on important technical directions, and she was usually the final decision-maker in internal review meetings.++He Meng joined CloudSail AI in 2023. After transitioning from lab to company, she was responsible for products and hospital delivery, with her desk close to Lin Zhiyao's office. Company employees say He Meng didn't need to knock because she often walked in with a stack of requirement documents, emerged ten-plus minutes later to send revision opinions to the project team. Although they now handle business decisions and product advancement separately, many colleagues can see that they still use the collaboration style from their lab days.++Another key figure on the technical side was **Chen Xi**. Chen Xi was a few years junior to Zhou Kai and had done engineering implementation in the lab early on. After joining CloudSail AI, he became the algorithm lead. When Zhou Kai held technical meetings, he often let Chen Xi report model performance first before supplementing resource arrangements. Many in the company knew that for anything involving training platforms and inference services, Zhou Kai typically went to Chen Xi first. Chen Xi didn't avoid this fact either. In an interview, he said many of his engineering habits today came from the period Zhou Kai mentored him.++**Su Man** was responsible for commercialization. She had previously worked on medical track research at an investment firm. After joining CloudSail AI, she visited almost all key national clients. When Su Man and Lin Zhiyao first traveled to Shanghai together, they discussed a hospital procurement list for two hours on the high-speed rail. Later, when CloudSail AI established pilot programs with several top-tier hospitals, much of the specific communication was done by Su Man and He Meng together: one responsible for clarifying clinical needs, the other for compressing the product schedule. Some joked internally that whenever Su Man and He Meng disappeared from their desks simultaneously, there was usually another hospital project about to be implemented.++CloudSail AI truly gained industry attention through its long-term collaboration with **Anlan Medical Group**. Anlan Medical Group CIO **Han Yicheng** first met Lin Zhiyao at a forum. After the forum ended, Han Yicheng didn't leave but chatted with her for another half hour outside the venue, discussing data governance and model auditing. A few months later, CloudSail AI began participating in Anlan Medical Group's assisted diagnosis platform construction. Since then, for a long period, He Meng and Han Yicheng's team held weekly project meetings, and even Anlan insiders knew that when facing difficulties with requirement consolidation, Han Yicheng would typically directly point to He Meng for coordination.++Although Prof. Shen Bowen didn't formally join CloudSail AI, he maintained contact with the company. When CloudSail AI conducted core algorithm reviews, they occasionally brought plans back to the university for Prof. Shen to review whether the experimental design was solid. In the company's third year, Lin Zhiyao took Zhou Kai back to Tsinghua University, and they chatted for an entire afternoon in the lab conference room with Prof. Shen. Later, Lin Zhiyao only briefly mentioned externally that it was a "defense-like" exchange.++However, the medical imaging AI track never belonged to just one startup. CloudSail AI's most common comparison was **Beichen Technology**. Beichen Technology founder **Zhao Yuxin** was similar in age to Lin Zhiyao. They appeared together at several industry forums annually but rarely publicly commented on each other directly. Several hospital IT directors mentioned that during the same project bidding cycle, CloudSail AI and Beichen Technology often entered sequentially for solution demonstrations, competing not just on model results but also on implementation team response speed.++Beichen Technology's Chief Architect **Gu Mingchuan** was also someone Zhou Kai paid special attention to. The two companies' technical teams had discussed inference efficiency and cost issues multiple times at industry closed-door meetings. Zhou Kai privately admitted that Gu Mingchuan was very skilled at system optimization. Externally, of course, he wouldn't say it so directly, but company engineers understood that whenever Zhou Kai got new publicly available materials from Beichen Technology, he would first send them to Chen Xi.++This competitive line also involved some old connections. Beichen Technology VP of Marketing **Tang Ying** had worked with He Meng at the same company early on, before each went separate ways. Although now belonging to competing companies, when they encountered each other at industry conferences, they would still chat briefly in the hallway, sometimes exchanging notes on event arrangements and client habits. Those who knew them well understood that behind this restrained small talk lay the tacit understanding of colleagues who had walked the same professional path for years.++Lin Zhiyao rarely uses the term "relationship network" in interviews. She prefers describing her work as a series of specific collaborations: building systems with Zhou Kai, pushing products with He Meng, clarifying clients with Su Man, and truly implementing projects in hospitals with people like Han Yicheng. But from an external perspective, what has formed around her over the past five years is no longer just a startup team, but a network connecting school, company, hospitals, and competitors through the people involved.diff --git a/tests/test_data/en/general/spatio_temporal_event_report.md b/tests/test_data/en/general/spatio_temporal_event_report.mdnew file mode 100644index 0000000..d6c81f1--- /dev/null+++ b/tests/test_data/en/general/spatio_temporal_event_report.md@@ -0,0 +1,103 @@+# 2024 Southern China Heavy Rain and Flood Disaster Rescue Operation Report++## Event Overview++From June 15 to 25, 2024, southern China experienced rare prolonged heavy rainfall, causing severe rainstorm and flood disasters in the middle and lower reaches of the Yangtze River and the Pearl River basin. This disaster had a wide impact, long duration, and strong destructive power, causing significant casualties and property losses.++## Disaster Overview++### Time Span++June 15 to July 10, 2024++### Affected Regions++- **Province A**: Cities B, C, D+- **Province E**: Cities F, G, H+- **Province I**: Cities J, K, L+- **Province M**: Cities N, O, P+- **Province Q**: Cities R, S, T++### Meteorological Data++On June 15, the Central Meteorological Observatory issued a red warning for heavy rain. This was the first river basin-level heavy rain red warning issued in nearly a decade.++## Rescue Operation Timeline++### Phase 1: Early Warning Response (June 15-17)++**June 15, 10:00 AM** - Beijing+The National Flood Control and Drought Relief Headquarters activated Level 2 emergency response, requiring all regions to take preventive measures.++**June 15, 2:00 PM** - City F+The Provincial E Flood Control and Drought Relief Command held an emergency meeting to deploy provincial flood control work. City F's Flood Control Command activated Level 3 emergency response.++**June 16, 8:00 AM** - All Affected Provinces+Provinces A, E, I, M, and Q all activated Level 1 emergency response. Armed police forces and militia reserve forces entered standby status.++**June 17, 6:00 AM** - City C+A scenic area in Province A's City C was urgently closed. Water levels at a section of the Xiang River reached the warning level of 36 meters.++### Phase 2: Emergency Rescue (June 18-22)++**June 18, 3:00 AM** - City D+A levee breach occurred in a district of City D. A unit from the PLA Central Theater urgently deployed 2,000 officers and soldiers to transfer 15,000 trapped residents overnight.++**June 18, 10:30 AM** - City J+Severe waterlogging occurred in a district of Province I's City J. Armed Police Provincial I Corps deployed 500 officers and soldiers with 30 assault boats for rescue operations.++**June 19, 8:00 AM** - City F+A levee seepage incident occurred in a district of City F. Two hundred personnel from the Army Engineering University's professional flood control and rescue force urgently rushed to the scene and successfully handled the emergency.++**June 20, 2:00 AM** - City O+During the transfer of trapped residents in a county of Province M's City O, an assault boat capsized, causing 3 residents to fall into the water. Armed Police Provincial M Corps officers and soldiers immediately jumped into the water to rescue them. All were rescued.++**June 21, 3:00 PM** - City S+A landslide occurred in a district of Province Q's City S, causing residential building collapse. Armed Police Provincial Q Corps deployed 100 officers and soldiers for search and rescue of buried residents.++**June 22, 9:00 AM** - City K+A town in Province I's City K became isolated, with over 2,000 residents trapped. An Air Force unit deployed 6 helicopter sorties to airdrop 5 tons of rescue supplies.++### Phase 3: Post-Disaster Resettlement (June 23 - July 5)++**June 23, 8:00 AM** - All Affected Areas+The Ministry of Emergency Management activated Level 4 disaster relief emergency response and arranged RMB 1 billion in central disaster relief funds.++**June 24, 10:00 AM** - City B+Province A established a temporary settlement at an exhibition center, accommodating 3,000 disaster-affected residents. Social donations of supplies arrived successively.++**June 26, 2:00 PM** - City F+Province E established a temporary settlement at a sports center, accommodating 4,500 disaster-affected residents.++**June 28, 8:00 AM** - City J+Province I established a temporary settlement at a gymnasium, accommodating 2,800 disaster-affected residents.++**July 1, 9:00 AM** - All Affected Areas+The National Health Commission organized medical and epidemic prevention teams to carry out medical treatment and sanitation and epidemic prevention work in disaster areas.++**July 3, 10:00 AM** - City D+Province A carried out post-disaster reconstruction planning for the most severely affected district in City D and initiated housing reconstruction work.++### Phase 4: Recovery and Reconstruction (July 6-10)++**July 6, 8:00 AM** - All Affected Provinces+The National Development and Reform Commission issued RMB 1.5 billion in central budget investment for post-disaster recovery and reconstruction.++**July 8, 9:00 AM** - City F+Province E held a post-disaster reconstruction work conference and identified 58 key restoration projects.++**July 10, 2:00 PM** - Beijing+The State Council held an executive meeting to deploy post-disaster recovery and reconstruction work, requiring efforts to restore normal production and living order within one year.++## Rescue Force Deployment++### PLA and Armed Police Forces++| Date | Location | Personnel | Main Tasks |+|------|---------|-----------|------------|+| June 18 | City D | 2,000 | Levee emergency repair, resident transfer |+| June 18 | City J | 500 | Waterlogging drainage, material transport |+| June 19 | City F | 200 | Levee reinforcement, emergency handling |+| June 20 | City O | 150 | Search and rescue of trapped residents |+| June 21 | City S | 100 | Landslide search and rescue |+| June 22 | City K | 6 helicopter sorties | Island rescue, material airdrop |diff --git a/tests/test_data/en/industry/equipment_technical_manual.md b/tests/test_data/en/industry/equipment_technical_manual.mdnew file mode 100644index 0000000..1b74ccc--- /dev/null+++ b/tests/test_data/en/industry/equipment_technical_manual.md@@ -0,0 +1,101 @@+# Factory A Equipment System Technical Manual++**Document Number**: ETM-2024-001+**Version**: v1.5+**Preparation Date**: January 2024+**Department**: Factory A++---++## 1. Factory Overview++Factory A is a modern manufacturing facility under the company, located within the industrial park, covering an area of approximately 50,000 square meters. The factory primarily engages in electronic product manufacturing and precision machining business.++### 1.1 Factory Scale++| Item | Data |+|------|------|+| Land Area | 50,000 m² |+| Building Area | 35,000 m² |+| Number of Employees | Approximately 500 |+| Annual Output | 500 million yuan |++### 1.2 Plant Layout++Factory A consists of the following main functional areas:++**Production Areas**:+- Manufacturing Workshop A: Automated assembly line (8,000 m²)+- Manufacturing Workshop B: Precision machining center (6,000 m²)+- Manufacturing Workshop C: Assembly workshop (5,000 m²)++**Supporting Areas**:+- Warehouse: Raw material and finished product storage (10,000 m²)+- Power Station: Power, water, and gas supply (2,000 m²)+- Office Area: Administrative offices (3,000 m²)++---++## 2. Production Systems++### 2.1 Manufacturing Workshop A - Automated Assembly Line++Manufacturing Workshop A is the core production workshop, mainly containing the following systems:++**CNC Machining Systems**:+- CNC Machining Center Alpha (TM-5000X, 2 units): Precision parts machining+- CNC Machining Center Beta (TM-5000X, 2 units): Precision parts machining++**Robot Systems**:+- Industrial Robot Arm R1 (AR-200, 3 units): Workpiece handling and assembly+- Industrial Robot Arm R2 (AR-200, 3 units): Workpiece handling and assembly+- Industrial Robot Arm R3 (AR-150, 2 units): Backup robot arms++**Material Handling Systems**:+- Conveyor Belt System (CT-500, 1 set): Workpiece transportation+- Automated Guided Vehicle AGV (LM-1000, 4 units): Material distribution++**Supporting Systems**:+- Cooling System: Provides cooling for CNC equipment+- Lubrication System: Provides lubrication for mechanical equipment+- Hydraulic System: Provides power for pneumatic equipment+- Dust Removal System: Maintains workshop cleanliness++### 2.2 Manufacturing Workshop B - Precision Machining Center++**Turning Systems**: 4 CNC lathes+**Milling Systems**: 3 CNC milling machines+**Grinding Systems**: 2 precision grinders+**Inspection Systems**: Coordinate measuring machine++### 2.3 Manufacturing Workshop C - Assembly Workshop++**Final Assembly Line**: Product final assembly+**Testing Line**: Product functional testing+**Packaging Line**: Product packaging and storage++---++## 3. Utility Systems++### 3.1 Power Supply System+- Transformer capacity: 2,000 KVA+- Supply voltage: 10 KV+- Backup power: Diesel generator 500 KW++### 3.2 Water Supply System+- Daily water supply: 200 tons+- Water supply pressure: 0.3 MPa++### 3.3 Gas Supply System+- Air compressors: 3 units+- Supply pressure: 0.7 MPa++---++## 4. Equipment List and Topology++### 4.1 Core Production Equipment++| Equipment ID | Equipment Name | Type | Location | Status | Manufacturer | Model |+|---|---|---|---|---|---|---|diff --git a/tests/test_data/en/industry/failure_analysis_report.md b/tests/test_data/en/industry/failure_analysis_report.mdnew file mode 100644index 0000000..277de90--- /dev/null+++ b/tests/test_data/en/industry/failure_analysis_report.md@@ -0,0 +1,79 @@+# Equipment Failure Case Analysis Report++**Report Number**: FA-2024-001+**Preparation Date**: March 15, 2024+**Equipment Name**: Automated Assembly Line A1 - Main Motor+**Failure Date**: January 10, 2024++---++## 1. Failure Overview++### 1.1 Equipment Information++| Item | Content |+|------|--------|+| Equipment ID | MTR-001 |+| Equipment Name | Main Motor |+| Model | DM-750 |+| Manufacturer | Power Drive Co., Ltd. |+| Installation Location | Manufacturing Workshop A, Zone 1 |+| Commissioning Date | June 2022 |++### 1.2 Failure Phenomenon++At 9:30 AM on January 10, 2024, the main motor (MTR-001) of Automated Assembly Line A1 suddenly emitted abnormal sounds during operation, accompanied by significantly increased vibration. The operator immediately pressed the emergency stop button, and the equipment stopped running.++**Specific Phenomena**:+- Motor emitted sharp metal friction sound+- Vibration amplitude exceeded normal value by 3 times+- Bearing temperature rapidly rose to 85°C+- Inverter displayed abnormal current fluctuation++---++## 2. Failure Analysis++### 2.1 Initial Inspection++After arriving at the site, maintenance personnel conducted initial inspection:++1. **Visual Inspection**: No obvious damage to motor housing, no abnormal odor+2. **Insulation Test**: Winding insulation resistance normal, ruled out short circuit possibility+3. **Vibration Test**: Vibration value exceeded standard, mainly concentrated at bearing area+4. **Temperature Test**: Bearing seat temperature was high at 85°C++### 2.2 In-depth Analysis++After disassembly and inspection, the root cause of the failure was confirmed as follows:++**Main Cause**: Severe bearing wear++- Bearing raceway showed obvious wear marks+- Rolling element surface had spalling phenomenon+- Lubricating grease turned black and deteriorated, losing lubrication function++**Secondary Cause**: Long-term lack of preventive maintenance++- The motor had not undergone bearing replacement since commissioning+- Lubricating grease was not regularly supplemented as required+- Operating load had been at high-load state for a long time++---++## 3. Handling Measures++### 3.1 Emergency Handling++1. Immediately stopped equipment operation and cut off power+2. Marked fault status and hung warning signs+3. Started backup motor to maintain production++### 3.2 Root Cause Treatment++1. **Bearing Replacement**: Replaced bearing with same specification (Model: 6208-2RS)+2. **Lubricating Grease Supplementation**: Used high-quality lubricating grease (Model: L-XBCHA3)+3. **Bearing Seat Cleaning**: Thoroughly removed old lubricating grease and impurities+4. **Electrical Testing**: Tested winding resistance to ensure normal electrical performance++### 3.3 Treatment Resultsdiff --git a/tests/test_data/en/industry/operation_maintenance_manual.md b/tests/test_data/en/industry/operation_maintenance_manual.mdnew file mode 100644index 0000000..6e85e67--- /dev/null+++ b/tests/test_data/en/industry/operation_maintenance_manual.md@@ -0,0 +1,80 @@+# Automated Assembly Line A1 Operation and Maintenance Manual++**Document Number**: OMM-2024-001+**Version**: v2.5+**Preparation Date**: March 2024+**Applicable Equipment**: Automated Assembly Line A1++---++## Part 1: Equipment Operation++### Chapter 1: Scope of Application++This procedure applies to the startup operation and operation management of Automated Assembly Line A1, including CNC machining centers, robot arm systems, material handling systems, and quality inspection stations.++---++### Chapter 2: Startup Operation Procedure++#### 2.1 Pre-Startup Inspection++**Environmental Inspection**:+- Temperature: Ambient temperature should be within 15-30°C+- Humidity: Relative humidity should be within 40-70%+- Cleanliness: No debris or water accumulation around equipment++**Equipment Inspection**:+- Power: Main power voltage normal, three-phase balanced+- Air Source: Air pressure maintained at 0.5-0.7 MPa+- Lubrication: All lubrication points have sufficient grease+- Safety Devices: Emergency stop button, protective door lock normal++#### 2.2 Startup Steps++**Step 1: Enable Auxiliary Systems**+- Operation: Turn on control cabinet auxiliary power+- Steps: Open control cabinet door → Close auxiliary power switch → Confirm auxiliary power indicator light is on+- Expected Result: Auxiliary power normally connected, control panel shows ready status+- Trigger Condition: Auxiliary power normal++**Step 2: Start Hydraulic System**+- Operation: Start hydraulic station+- Steps: Press hydraulic station start button → Wait for hydraulic pressure to rise to set value → Check: Hydraulic pressure stable at 3.5 MPa, oil level normal+- Expected Result: Hydraulic pressure stable at 3.5 MPa, oil level normal+- Trigger Condition: Hydraulic pressure reaches set value++**Step 3: Start Cooling System**+- Operation: Turn on cooling pump+- Steps: Turn on cooling pump power switch → Adjust coolant flow rate → Check coolant temperature+- Expected Result: Cooling system operates normally, inlet-outlet temperature difference is 5-10°C+- Trigger Condition: Cooling pump operates without abnormality++**Step 4: Start Control System**+- Operation: Start main control system+- Steps: Select "Auto Mode" on HMI → Click "System Start" button → Wait for system self-check to complete+- Expected Result: HMI displays "System Ready," no alarm information+- Trigger Condition: System self-check passed++**Step 5: Activate Servo System**+- Operation: Start servo drive+- Steps: Start each axis servo drive in sequence → Wait for servo system to be ready → Execute origin return+- Expected Result: Each axis returns to reference origin, position display correct+- Trigger Condition: Servo system has no faults++**Step 6: Start Material Handling System**+- Operation: Turn on conveyor and AGV+- Steps: Start conveyor motor → Activate AGV navigation system → Set transport path+- Expected Result: Conveyor operates normally, AGV in position on standby+- Trigger Condition: Conveyor operates smoothly++**Step 7: Start Robot System**+- Operation: Activate robot arm+- Steps: Start robot arm control cabinet → Load processing program → Execute robot arm origin return+- Expected Result: All robot arm joints normal, arriving at origin position+- Trigger Condition: Robot arm has no collision alarm++**Step 8: Start Quality Inspection System**+- Operation: Turn on inspection equipment+- Steps: Start vision inspection system → Calibrate inspection parameters → Load inspection program+- Expected Result: Inspection system ready, can normally identify workpiecesdiff --git a/tests/test_data/en/industry/safety_management_handbook.md b/tests/test_data/en/industry/safety_management_handbook.mdnew file mode 100644index 0000000..80a19e9--- /dev/null+++ b/tests/test_data/en/industry/safety_management_handbook.md@@ -0,0 +1,79 @@+# Automated Assembly Line A1 Safety Management and Emergency Response Manual++**Document Number**: SMC-2024-001+**Version**: v2.0+**Preparation Date**: February 2024+**Scope**: All operators of Automated Assembly Line A1++---++## Part 1: Safety Control++### Chapter 1: General Provisions++#### 1.1 Purpose++This manual is specially formulated to strengthen safety production management of Automated Assembly Line A1, prevent and reduce production safety accidents, and protect employees' lives, safety and physical health.++#### 1.2 Scope of Application++This manual applies to all operational activities of Automated Assembly Line A1, including equipment operation, maintenance, inspection, etc.++---++### Chapter 2: Hazard Identification++#### 2.1 Equipment Hazards++| No. | Hazard Name | Hazard Factor | Possible Consequences |+|-----|-------------|--------------|----------------------|+| 1 | CNC Machining Center | High temperature components, rotating parts | Burns, mechanical injury |+| 2 | Industrial Robot Arm | Moving parts, clamping force | Crushing injury, impact injury |+| 3 | Cooling System | High-pressure liquid, temperature changes | Burns, frostbite |+| 4 | Hydraulic Station | High-pressure oil, seal aging | High-pressure jet injury |+| 5 | Electrical Control Cabinet | Electrical components, high-voltage power | Electric shock, electrocution |++#### 2.2 Environmental Hazards++| No. | Hazard Name | Hazard Factor | Possible Consequences |+|-----|-------------|--------------|----------------------|+| 1 | Workshop Aisles | Ground oil stains, object stacking | Slips, trips |+| 2 | Material Stacking Area | Material toppling, instability | Crushing injury, collisions |+| 3 | Noise Areas | Equipment operation noise | Hearing damage |+| 4 | Chemical Storage | Toxic and hazardous chemicals | Poisoning, chemical burns |++#### 2.3 Behavioral Hazards++| No. | Hazard Name | Hazard Factor | Possible Consequences |+|-----|-------------|--------------|----------------------|+| 1 | Violation of Regulations | Violating operating procedures | Equipment damage, personal injury |+| 2 | Fatigue Work | Continuous work, insufficient rest | Slow reactions, operational errors |+| 3 | Lack of Protection | Not wearing protective equipment | Personal injury |++---++### Chapter 3: Risk Points and Control Measures++#### 3.1 Mechanical Equipment Risks++| Risk Point | Risk Level | Control Measures | Responsible Person |+|------------|-----------|-----------------|-------------------|+| CNC machining center operation | Major risk | 1. Strictly follow operating procedures 2. Wear safety glasses and protective gloves 3. Prohibit opening protective doors while equipment is running 4. Regularly check effectiveness of protective devices | Equipment supervisor |+| Robot arm operation area | Major risk | 1. Set safety fences and warning signs 2. Install light curtain protection devices 3. Confirm no personnel in safety area before operation 4. Strictly prohibit unauthorized entry into operation area | Shift leader |+| Hydraulic system maintenance | High risk | 1. Release pressure before maintenance 2. Wear protective gloves and safety glasses 3. Use professional tools for maintenance 4. Conduct pressure test after maintenance | Maintenance worker |++#### 3.2 Electrical Safety Risks++| Risk Point | Risk Level | Control Measures | Responsible Person |+|------------|-----------|-----------------|-------------------|+| Electrical cabinet operation | Major risk | 1. Only qualified electrical workers may operate 2. Perform voltage check before operation 3. Wear insulated shoes and gloves 4. Prohibit live work | Electrician |+| Temporary electricity use | High risk | 1. Complete temporary electricity approval 2. Use qualified electrical equipment 3. Regularly check line insulation performance 4. Implement one-machine-one-switch-one-protection | Equipment supervisor |+| Static electricity protection | General risk | 1. Wear anti-static work clothes 2. Ensure reliable equipment grounding 3. Maintain appropriate ambient humidity | Safety officer |++#### 3.3 Chemical Safety Risks++| Risk Point | Risk Level | Control Measures | Responsible Person |+|------------|-----------|-----------------|-------------------|+| Coolant use | High risk | 1. Wear protective gloves and safety glasses 2. Keep workplace ventilated 3. Avoid direct skin contact 4. Regularly check coolant concentration | Equipment supervisor |+| Lubricating grease | General risk | 1. Wear protective gloves 2. Avoid contact with mouth and eyes 3. Use special tools for adding 4. Unified recycling of waste oil | Maintenance worker |+| Cleaning agent use | High risk | 1. Use in ventilated areas 2. Wear gas masks 3. Store away from fire sources 4. Operate according to MSDS requirements | Shift leader |diff --git a/tests/test_data/en/legal/compliance_filing_sample.md b/tests/test_data/en/legal/compliance_filing_sample.mdnew file mode 100644index 0000000..019e218--- /dev/null+++ b/tests/test_data/en/legal/compliance_filing_sample.md@@ -0,0 +1,81 @@+# XX Technology Group Co., Ltd.++# 2024 Annual Anti-Money Laundering and Counter-Terrorist Financing Compliance Report++**Reporting Period**: January 1, 2024 to December 31, 2024+**Filing Date**: March 15, 2025+**Filing Institution**: XX City Central Branch of the People's Bank of China+**Document Number**: AML-2024-XXX-001++---++## Part 1: Basic Company Information++### 1.1 Company Overview++| Item | Content |+|---|---|+| Company Name | XX Technology Group Co., Ltd. |+| Unified Social Credit Code | 91310000MA1XXXXXXX |+| Registered Capital | RMB XX billion |+| Registered Address | XX Park, XX District, XX City |+| Date of Incorporation | March 2010 |+| Main Business | Financial technology software R&D, intelligent investment advisory services, payment and clearing |+| Financial License Number | B0001H231000001 |++### 1.2 Shareholding Structure Overview++As of December 31, 2024, the Company's shareholding structure is as follows:++| No. | Shareholder Name | Shares Held (10,000) | Shareholding Ratio | Shareholder Type |+|---|---|---|---|---|+| 1 | XX Holding Group Co., Ltd. | XX,XXX | XX.XX% | Domestic Legal Entity |+| 2 | Innovation Works Investment Fund (LP) | XX,XXX | XX.XX% | Domestic Fund |+| 3 | Junzheng Capital Management Co., Ltd. | XX,XXX | XX.XX% | Domestic Legal Entity |+| 4 | Shenzhen Qianhai Innovation Investment | X,XXX | X.XX% | Domestic Legal Entity |+| 5 | Employee Stock Ownership Platform (LP) | X,XXX | X.XX% | Domestic Limited Partnership |+| 6 | Overseas Investor BVI Company | XX,XXX | XX.XX% | Overseas Legal Entity |+| 7 | Other Public Shareholders | XX,XXX | XX.XX% | Public Shareholders |++---++## Part 2: Beneficial Ownership Identification++### 2.1 Equity Penetration Analysis++In accordance with the requirements of the "Administrative Measures for Financial Institutions' Large-Value Transactions and Suspicious Transaction Reports" and the "Notice of the People's Bank of China on Strengthening Customer Identity Identification in Anti-Money Laundering," the Company has conducted equity penetration analysis on shareholders holding more than 5% equity, as detailed below:++#### 2.1.1 XX Holding Group Co., Ltd. (XX% shareholding)++**Equity Structure**:++| Level | Shareholder Name | Shareholding Ratio | Type |+|---|---|---|---|+| 1 | Michael Chen | XX% | Natural Person |+| 1 | Jennifer Wang | XX% | Natural Person |+| 1 | Steven Lin | XX% | Natural Person |++**Ultimate Beneficial Owners**: Michael Chen (XX% shareholding), Jennifer Wang (XX% shareholding), Steven Lin (XX% shareholding)++**Control Relationship Description**: Michael Chen serves as Executive Director of XX Holding Group Co., Ltd. and has actual control over the Company's business decisions.++---++#### 2.1.2 Innovation Works Investment Fund (LP) (XX% shareholding)++**General Partner**: Innovation Works Asset Management (XX City) Co., Ltd.++**Equity Structure**:++| Level | Shareholder Name | Shareholding Ratio | Type |+|---|---|---|---|+| 1 | Innovation Works Asset Management (XX City) Co., Ltd. | 1% | Domestic Legal Entity |+| 2 | Sarah Li | XX% | Natural Person |+| 2 | Kevin Wang | XX% | Natural Person |++**Limited Partners**:++| Partner Name | Capital Contribution (RMB 10,000) | Type |+|---|---|+| XX City Angel Investment Guidance Fund | XX,XXX | Government Investment |+| National Social Security Fund Council | XX,XXX | State-owned Capital |diff --git a/tests/test_data/en/legal/court_judgment_sample.md b/tests/test_data/en/legal/court_judgment_sample.mdnew file mode 100644index 0000000..02c3cf9--- /dev/null+++ b/tests/test_data/en/legal/court_judgment_sample.md@@ -0,0 +1,81 @@+# People's Court of XX District, XX City++# Civil Judgment++# (2024) Hu 0115 Min Chu No. 12345++---++## Party Information++**Plaintiff**: XX Venture Capital Investment Co., Ltd.+Unified Social Credit Code: 91310000MA1XXXXXXX+Address: XX Road, XX District, XX City+Legal Representative: Sarah Li, Executive Director+Authorized Litigation Representative: Michael Wang, Attorney at XX Law Firm++**Defendant**: YY Technology Co., Ltd.+Unified Social Credit Code: 91310000MA2XXXXXXX+Address: XX Road, XX District, XX City+Legal Representative: David Zhang, General Manager+Authorized Litigation Representative: Jennifer Zhao, Attorney at XX Law Firm++**Third Party**: ZZ Capital Management Co., Ltd.+Unified Social Credit Code: 91440300MA3XXXXXXX+Address: XX Road, XX District, XX City+Legal Representative: Michelle Chen, Chairman++---++## Case Details++**Cause of Action**: Shareholder Capital Contribution Dispute+**Trial Procedure**: Ordinary Procedure+**Panel Composition**: Presiding Judge Richard Liu, Judge William Zhou, People's Assessor Jennifer Zheng+**Court Clerk**: Michael Zhang++---++## Findings of Fact++### 1. Relationship Between Parties++XX Venture Capital Investment Co., Ltd. (Plaintiff) and YY Technology Co., Ltd. (Defendant) signed a "Strategic Cooperation Agreement" in March 2021, agreeing that Plaintiff would invest RMB XXXX million in Defendant and hold 15% equity. ZZ Capital Management Co., Ltd. (Third Party) is the original shareholder of Defendant, holding 10% equity.++### 2. Timeline of Case Facts++**March 15, 2021**: Plaintiff and Defendant signed "Strategic Cooperation Agreement," agreeing that Plaintiff would provide RMB XXXX million investment for XX software platform R&D. Article 5 of the Agreement states: Within 90 days after the end of each fiscal year, Defendant shall distribute cash to shareholders at 30% of distributable profits for that year, and Plaintiff shall enjoy investment income according to its shareholding ratio.++**April 1, 2021**: Plaintiff transferred investment funds of RMB XXXX million to Defendant's designated account via bank transfer.++**May 10, 2021**: Plaintiff and Defendant signed a "Supplementary Agreement," agreeing to adjust the income distribution ratio in Article 5 of the original "Strategic Cooperation Agreement" to: Plaintiff may only obtain 20% of the excess portion as investment income when Defendant's annual net profit exceeds RMB XXXX million. This agreement was not passed through Defendant's shareholders' meeting resolution.++**June 20, 2021**: Defendant completed industrial and commercial registration changes, and Plaintiff formally held 15% equity in Defendant.++**January 10, 2022**: Defendant held a shareholders' meeting, resolving the 2021 profit distribution plan: distribute dividends to shareholders at 30% of net profit.++**March 5, 2022**: Defendant distributed dividends of RMB XX million to Plaintiff. Plaintiff believed that according to the "Supplementary Agreement," Defendant's 2021 net profit was RMB XXXX million, which did not exceed the RMB XXXX million threshold, so Plaintiff was not entitled to income distribution, and the RMB XX million was a friendly gesture.++**August 15, 2022**: Defendant sent a "Profit Distribution Explanation Letter" to Plaintiff, claiming that according to the "Supplementary Agreement," Plaintiff was not entitled to fixed-proportion income for 2021 and only received a dividend of RMB XX million.++**January 20, 2023**: Plaintiff filed a lawsuit with this Court, requesting confirmation of the invalidity of the "Supplementary Agreement" and demanding Defendant pay outstanding investment income and liquidated damages.++---++## Issues in Dispute++### Issue 1: Validity of the "Supplementary Agreement"++**Plaintiff's Claim**: The "Supplementary Agreement" is a true expression of both parties' intentions and is legally valid. As an investor, Plaintiff negotiated with Defendant to adjust the income distribution method, which falls within the scope of business judgment and does not violate legal provisions.++**Defendant's Claim**: The "Supplementary Agreement" was signed through mutual negotiation and agreement, representing a new arrangement of investment income between both parties, and is legally valid.++**Court's Determination**: According to Article 8 of Defendant's articles of association, major agreement changes between the Company and external investors must be passed through special shareholders' meeting resolution (approved by shareholders holding more than two-thirds of shares). The "Supplementary Agreement" was not passed through Defendant's shareholders' meeting resolution, violating Defendant's internal articles of association. Additionally, the "Supplementary Agreement" adjusted Plaintiff's income distribution conditions from a fixed ratio to excess decrement, substantially reducing Plaintiff's investment income and potentially harming Defendant and other shareholders' interests.++**Applicable Law**: Article 154 of the Civil Code of the People's Republic of China: A civil legal act where the actor and the counterparty maliciously collude to harm the legitimate rights and interests of others is invalid.++**Conclusion**: Confirmed that the "Supplementary Agreement" is invalid.++---++### Issue 2: Whether Defendant Committed Breach of Contractdiff --git a/tests/test_data/en/legal/legal_treatise_sample.md b/tests/test_data/en/legal/legal_treatise_sample.mdnew file mode 100644index 0000000..7df8ef2--- /dev/null+++ b/tests/test_data/en/legal/legal_treatise_sample.md@@ -0,0 +1,80 @@+# Principles of Fault Determination in Civil Tort Liability — Centered on Article 1165 of the Civil Code++## Chapter 1: Basic Theory of Tort Liability Law++### 1.1 Jurisprudential Basis of Fault Liability Principle++The fault liability principle is the core attribution principle of civil tort liability. Its jurisprudential basis lies in:+- **Principle of Individual Responsibility**: Actors are only responsible for their own wrongful acts+- **Preventive Function**: Through sanctions against wrongful acts, preventing the occurrence of improper behavior+- **Freedom Guarantee**: Actors enjoy freedom of action without violating laws++### 1.2 Concept and Types of Fault++**Fault** includes two forms: intent and negligence:++**Intent**: The psychological state of an actor who foresees the occurrence of damage and hopes or allows the result to occur. Intent can be divided into:+- Direct intent: hoping for the occurrence of damage+- Indirect intent: allowing the occurrence of damage++**Negligence**: The psychological state where an actor should have foreseen the occurrence of damage but failed to foresee it due to carelessness, or had foreseen it but believed it could be avoided. Negligence can be divided into:+- Ordinary negligence: duty of care of an ordinary person+- Gross negligence: clearly exceeding the degree of care of an ordinary person++---++## Chapter 2: Normative Interpretation of Article 1165 of the Civil Code++### 2.1 Text Content++> Article 1165 of the Civil Code of the People's Republic of China: An actor who causes damage to another person's civil rights and interests due to fault shall bear tort liability.+> If fault is presumed against an actor according to law, and the actor cannot prove the absence of fault, they shall bear tort liability.++### 2.2 Normative Structure++This article adopts the legislative model of a general clause, containing the following elements:+1. **Act**: An act that infringes upon civil rights and interests+2. **Fault**: Intent or negligence in the actor's subjectivity+3. **Damage**: The occurrence of actual damage+4. **Causal Relationship**: Existence of a causal relationship between the act and the damage++### 2.3 Burden of Proof++According to the "whoever claims must prove" principle, the plaintiff must prove:+- The defendant committed a tortious act+- The defendant had fault+- The plaintiff suffered damage+- There is a causal relationship between the act and the damage++Exception: When the law presumes fault, the defendant must prove the absence of their fault (e.g., product liability, medical malpractice liability).++---++## Chapter 3: Standards for Fault Determination in Case Law++### 3.1 Fault Determination in Cases of Security Obligation++**Guiding Case 140**: In public places or roads, excavation, repair, or installation of underground facilities causing damage to others, if the constructor cannot prove that obvious signs have been set up and safety measures have been taken, they shall bear tort liability.++This case establishes the "result obligation" nature of the security obligation — as long as damage occurs and obligation fulfillment cannot be proven, fault is presumed.++### 3.2 Fault Determination in Medical Malpractice Liability++**Guiding Case 142**: If a patient suffers damage during diagnosis and treatment activities, and the medical institution and its medical personnel have fault, the medical institution shall bear compensation liability. The determination of medical fault shall be based on the medical standard at that time.++Here, "medical standard at that time" includes:+- Duty of care stipulated by diagnosis and treatment norms+- Medical consensus at that time+- The conditions and capabilities of the medical institution itself++### 3.3 Fault Presumption in High-Risk Liability++According to Article 1239 of the Civil Code, operators engaged in high-altitude, high-pressure, underground excavation activities or using high-speed rail transportation tools causing damage to others shall bear tort liability; however, if they can prove that the damage was caused by the victim's intentional act or force majeure, they shall not bear liability.++---++## Chapter 4: Doctrinal Debates on Fault Determination++### 4.1 Subjective Fault Theory vs. Objective Fault Theory++**Subjective Fault Theory**: Fault is the internal psychological state of an actor, constituting moral condemnation.diff --git a/tests/test_data/en/legal/msa_contract_sample.md b/tests/test_data/en/legal/msa_contract_sample.mdnew file mode 100644index 0000000..606d7d1--- /dev/null+++ b/tests/test_data/en/legal/msa_contract_sample.md@@ -0,0 +1,99 @@+# Master Services Agreement for Software Development and Technical Services++**Agreement Number**: MSA-2024-XXX-001+**Effective Date**: January 15, 2024++---++## Parties++**Party A (Client)**: XX Technology Co., Ltd.+Unified Social Credit Code: 91310000MA1XXXXXXX+Registered Address: XX Road, XX District, XX City+Legal Representative: John Zhang++**Party B (Service Provider)**: YY Software Technology Co., Ltd.+Unified Social Credit Code: 91110000MA2XXXXXXX+Registered Address: XX Park, XX District, XX City+Legal Representative: Sarah Li++---++## Article 1: Definitions and Interpretations++1.1 **"Project"**: Refers to the XX Management System development project jointly implemented by both parties.++1.2 **"Software Product"**: Refers to the XX Management System v2.0 developed by Party B for Party A as agreed in this Agreement, including front-end interfaces, back-end services, database, and mobile applications.++1.3 **"Technical Documentation"**: Refers to design documents, user manuals, installation guides, operation instructions, and API interface documentation related to the software product.++1.4 **"Intellectual Property"**: Refers to patent rights, copyrights, trademark rights, trade secrets, and other rights related to intellectual achievements.++1.5 **"Confidential Information"**: Refers to trade secrets, technical information, customer data, and business plans disclosed by one party to the other during cooperation that are not known to the public.++1.6 **"Force Majeure"**: Refers to unforeseeable, unavoidable, and insurmountable objective circumstances, including but not limited to natural disasters, war, strikes, government actions, and cyber attacks.++1.7 **"Acceptance Criteria"**: Refers to the functional, performance, and quality requirements of the software product as agreed by both parties, detailed in Appendix "Requirements Specification."++1.8 **"Service Fee"**: Refers to the project development and technical service fees paid by Party A to Party B as agreed in this Agreement, with a total amount of RMB XXX million.++---++## Article 2: Scope and Content of Services++2.1 **System Development**: Party B shall develop XX Management System v2.0, including the following modules:+ - Production Planning and Scheduling Module+ - Equipment Management Module+ - Quality Traceability Module+ - Data Analysis Dashboard Module++2.2 **Technical Training**: Party B shall provide Party A with no less than 40 person-hours of technical training after software delivery.++2.3 **Operation and Maintenance Services**: During the quality assurance period (12 months from the date of acceptance), Party B shall provide free operation and maintenance services, including:+ - Defect fixes+ - Security patch updates+ - Performance optimization++2.4 **Secondary Development**: During the operation and maintenance period, Party B shall cooperate with Party A for secondary development work not exceeding 200 person-hours.++---++## Article 3: Project Schedule and Milestones++3.1 **Phase 1**: Requirements Analysis and Design (January - March 2024)+ - Complete "Requirements Specification"+ - Complete system architecture design+ - Deliverables: Requirements documents, architecture design documents++3.2 **Phase 2**: Core Functionality Development (April - August 2024)+ - Complete core module development+ - Complete internal testing+ - Deliverables: Runnable software, internal test reports++3.3 **Phase 3**: Integration and Acceptance (September - November 2024)+ - Complete system integration+ - Complete UAT testing+ - Deliverables: Acceptance test report, complete technical documentation++3.4 **Phase 4**: Go-Live and Quality Assurance (December 2024 - November 2025)+ - System go-live and deployment+ - Enter quality assurance period+ - Provide operation and maintenance services++---++## Article 4: Fees and Payment Terms++4.1 **Total Service Fee**: RMB XXX million (including 6% VAT)++4.2 **Payment Milestones**:+ - Initial Payment: Within 5 working days after contract signing, pay 30% of total contract amount, totaling RMB XXX million+ - Phase Payment: Within 5 working days after Phase 2 completion and Party A confirmation, pay 30% of total contract amount, totaling RMB XXX million+ - Acceptance Payment: Within 5 working days after system acceptance, pay 30% of total contract amount, totaling RMB XXX million+ - Quality Assurance Deposit: Within 5 working days after quality assurance period expiration, pay 10% of total contract amount, totaling RMB XXX million++4.3 **Payment Conditions**: Before Party A makes payment, Party B shall issue corresponding value-added tax special invoices.++---++## Article 5: Intellectual Property Ownershipdiff --git a/tests/test_data/en/medicine/clinical_guideline_sample.md b/tests/test_data/en/medicine/clinical_guideline_sample.mdnew file mode 100644index 0000000..f73b3fb--- /dev/null+++ b/tests/test_data/en/medicine/clinical_guideline_sample.md@@ -0,0 +1,100 @@+# Type 2 Diabetes Prevention and Treatment Guidelines (2020 Edition)++## 1. Overview++Type 2 Diabetes Mellitus (T2DM) is a common chronic metabolic disease characterized by insulin resistance and insufficient insulin secretion, leading to elevated blood glucose levels. This guideline aims to provide clinicians with standardized diagnosis and treatment recommendations to improve the prevention and treatment of type 2 diabetes.++## 2. Diagnostic Criteria++### 2.1 Diabetes Diagnostic Criteria+- Glycated hemoglobin (HbA1c) ≥6.5%+- Fasting plasma glucose (FPG) ≥7.0 mmol/L+- 2-hour plasma glucose during oral glucose tolerance test (OGTT) ≥11.1 mmol/L+- Random plasma glucose ≥11.1 mmol/L with classic hyperglycemia symptoms++### 2.2 Prediabetes Diagnostic Criteria+- Impaired fasting glucose (IFG): 6.1≤FPG<7.0 mmol/L+- Impaired glucose tolerance (IGT): 7.8≤OGTT 2-hour glucose<11.1 mmol/L+- Glycated hemoglobin: 5.7%≤HbA1c<6.5%++## 3. Treatment Pathway++### 3.1 Lifestyle Intervention+**Recommendation Grade**: A+**Evidence Level**: 1a++All type 2 diabetes patients should receive lifestyle intervention, including:+- Rational diet: Control total calories, properly distribute carbohydrates, protein, and fat+- Regular exercise: At least 150 minutes of moderate-intensity aerobic exercise per week+- Weight control: Keep BMI below 24 kg/m²+- Quit smoking and limit alcohol+- Psychological balance++### 3.2 Glycemic Control Targets+**Recommendation Grade**: A+**Evidence Level**: 1a++- General adults: HbA1c <7.0%+- Elderly patients (over 65): HbA1c <7.5% (good health) or <8.0% (poor health)+- Gestational diabetes: HbA1c <6.5%+- Patients with severe complications or high hypoglycemia risk: HbA1c <8.0%++### 3.3 Medication Treatment Pathway++#### 3.3.1 Monotherapy+**Recommendation Grade**: A+**Evidence Level**: 1a++- First-line: Metformin: Starting dose 500mg, 2-3 times daily, maximum dose 2000mg/day+- If metformin is intolerant, alternatives include:+ - Sulfonylureas: Glimepiride 1-4mg/day+ - Thiazolidinediones: Pioglitazone 15-45mg/day+ - DPP-4 inhibitors: Sitagliptin 100mg/day+ - SGLT-2 inhibitors: Dapagliflozin 10mg/day++#### 3.3.2 Dual Therapy+**Recommendation Grade**: A+**Evidence Level**: 1a++If HbA1c target not achieved after 3 months of monotherapy, initiate dual therapy:+- Metformin + Sulfonylureas+- Metformin + Thiazolidinediones+- Metformin + DPP-4 inhibitors+- Metformin + SGLT-2 inhibitors+- Metformin + GLP-1 receptor agonists++#### 3.3.3 Triple Therapy+**Recommendation Grade**: A+**Evidence Level**: 1b++If HbA1c target not achieved after 3 months of dual therapy, initiate triple therapy:+- Metformin + Sulfonylureas + Thiazolidinediones+- Metformin + Sulfonylureas + DPP-4 inhibitors+- Metformin + Sulfonylureas + SGLT-2 inhibitors+- Metformin + Sulfonylureas + GLP-1 receptor agonists++#### 3.3.4 Insulin Therapy+**Recommendation Grade**: A+**Evidence Level**: 1a++If HbA1c target not achieved after 3 months of triple therapy, initiate insulin therapy:+- Basal insulin: Insulin glargine, starting dose 0.1-0.2 U/kg/day+- Premixed insulin: Insulin aspart 30, starting dose 0.2-0.4 U/kg/day, injected before breakfast and dinner+- Basal-bolus insulin: Insulin aspart + insulin glargine, adjust dose according to blood glucose++## 4. Complication Screening and Management++### 4.1 Macrovascular Complications+**Recommendation Grade**: A+**Evidence Level**: 1a++- Cardiovascular disease: Annual ECG examination, assess cardiovascular risk+- Cerebrovascular disease: Regular carotid ultrasound examination+- Lower extremity vascular disease: Annual ankle-brachial index (ABI) examination++### 4.2 Microvascular Complications+**Recommendation Grade**: A+**Evidence Level**: 1a++- Diabetic kidney disease: Annual urine albumin/creatinine ratio (UACR) and serum creatinine examination+- Diabetic retinopathy: Annual fundus examinationdiff --git a/tests/test_data/en/medicine/discharge_summary_sample.md b/tests/test_data/en/medicine/discharge_summary_sample.mdnew file mode 100644index 0000000..6361866--- /dev/null+++ b/tests/test_data/en/medicine/discharge_summary_sample.md@@ -0,0 +1,90 @@+# Discharge Summary++## Basic Information+- **Patient Name**: Zhang San (John Smith)+- **Gender**: Male+- **Age**: 65 years old+- **Inpatient Number**: 20260201001+- **Admission Date**: February 1, 2026+- **Discharge Date**: February 10, 2026+- **Admission Diagnosis**: 1. Coronary Atherosclerotic Heart Disease 2. Unstable Angina 3. Hypertension Stage 3 (Very High Risk)+- **Discharge Diagnosis**: 1. Coronary Atherosclerotic Heart Disease 2. Unstable Angina 3. Hypertension Stage 3 (Very High Risk) 4. Type 2 Diabetes Mellitus++## Hospital Course++### February 1, 2026+- **Admission Condition**: Patient admitted due to "recurrent chest pain for 2 weeks, aggravated for 1 day." At admission, chest pain was crushing in nature, located in the precordial area, about the size of a palm, radiating to the left shoulder, lasting about 15 minutes, relieved by rest.+- **Physical Examination**: Temperature 36.5°C, pulse 72/min, respiration 18/min, blood pressure 160/95 mmHg. Heart border not enlarged, heart rate 72/min, regular rhythm, no murmurs heard in all valve auscultation areas.+- **Diagnostic Tests**: ECG: Sinus rhythm, ST-segment depression 0.1-0.2 mV in leads V1-V4. Blood routine, liver and kidney function, electrolytes normal.++### February 2, 2026+- **Tests**: Cardiac enzymes: Troponin I 0.05 ng/ml (normal), CK-MB 15 U/L (normal). Echocardiography: Left ventricular ejection fraction 60%, all cardiac chambers normal size, no obvious abnormal wall motion.+- **Treatment**: Prescribed aspirin 100mg qd, clopidogrel 75mg qd, atorvastatin 20mg qn, metoprolol 25mg bid, valsartan 80mg qd, isosorbide mononitrate 20mg bid.++### February 3, 2026+- **Tests**: Coronary angiography: Left main artery no significant stenosis, proximal LAD stenosis 80%, mid-LCX stenosis 50%, mid-RCA stenosis 40%.+- **Diagnosis**: Coronary atherosclerotic heart disease, unstable angina, three-vessel disease.++### February 4, 2026+- **Surgery**: Coronary Artery Bypass Grafting (CABG)+ - **Surgeons**: Dr. Wang (chief), Dr. Li (assistant)+ - **Surgery Time**: 08:30-14:30+ - **Surgical Method**: Off-pump coronary artery bypass grafting+ - **Grafts**: Left internal mammary artery → LAD, great saphenous vein → LCX, great saphenous vein → RCA+ - **Intraoperative Condition**: Surgery smooth, blood loss approximately 300ml, no transfusion required.++### February 5, 2026+- **Postoperative Condition**: Patient transferred to ICU, ventilator-assisted breathing, vital signs stable.+- **Treatment**: Dopamine 3μg/kg/min IV pump, nitroglycerin 5μg/min IV pump, cefuroxime 1.5g q8h IV drip, omeprazole 40mg q12h IV push.++### February 6, 2026+- **Postoperative Condition**: Patient extubated, changed to nasal cannula oxygen, oxygen saturation 98%.+- **Treatment**: Discontinued dopamine, continued nitroglycerin IV pump, adjusted to 3μg/min.++### February 7, 2026+- **Postoperative Condition**: Patient transferred to general ward, vital signs stable, no chest pain, dyspnea or other discomfort.+- **Tests**: Blood routine: WBC 8.5×10⁹/L, hemoglobin 120g/L, platelets 180×10⁹/L. Liver and kidney function, electrolytes normal.+- **Treatment**: Discontinued nitroglycerin IV pump, changed to oral isosorbide mononitrate 20mg bid.++### February 8, 2026+- **Tests**: Blood glucose: Fasting glucose 7.8 mmol/L, 2-hour postprandial glucose 11.2 mmol/L.+- **Diagnosis**: Type 2 diabetes mellitus.+- **Treatment**: Added metformin 500mg bid orally.++### February 9, 2026+- **Tests**: ECG: Sinus rhythm, ST-segment in V1-V4 improved compared to previous.+- **Treatment**: Adjusted metoprolol dose to 50mg bid orally.++### February 10, 2026+- **Discharge Assessment**: Patient's general condition good, no chest pain, dyspnea or other discomfort, vital signs stable. Wound healing well, no redness, swelling, or exudate.+- **Discharge Orders**: See discharge orders section for details.++## Discharge Orders++### Medication+- **Antiplatelet Drugs**: Aspirin 100mg qd, lifelong; Clopidogrel 75mg qd, for 12 months+- **Lipid-lowering Drugs**: Atorvastatin 20mg qn, lifelong+- **Beta-blockers**: Metoprolol 50mg bid, adjust dose according to heart rate+- **ACEI/ARB**: Valsartan 80mg qd, lifelong+- **Nitrates**: Isosorbide mononitrate 20mg bid, may gradually reduce when symptoms relieved+- **Antidiabetic Drugs**: Metformin 500mg bid, monitor blood glucose+- **Antibiotics**: Cefuroxime axetil 250mg bid, for 3 days++### Diet and Activity+- **Diet**: Low-salt, low-fat diabetic diet, control total calories, daily salt intake <5g+- **Activity**: Avoid strenuous exercise within 1 month after surgery, may engage in light activities such as walking; gradually resume normal activities after 3 months+- **Weight**: Control weight, target BMI <24 kg/m²++### Follow-up Plan+- **1 Week After Discharge**: Cardiology outpatient follow-up, recheck blood routine, liver and kidney function, electrolytes+- **2 Weeks After Discharge**: Wound suture removal, cardiothoracic surgery outpatient follow-up+- **1 Month After Discharge**: Cardiology outpatient follow-up, recheck ECG and echocardiography+- **3 Months After Discharge**: Cardiology outpatient follow-up, recheck coronary CTA+- **6 Months After Discharge**: Cardiology outpatient follow-up, comprehensive cardiac function assessment++### Precautions+- Keep wound clean and dry, avoid infection+- Take medication on time, do not increase or decrease dosage on your own+- Regularly monitor blood pressure and blood glucose+- If chest pain, dyspnea, fever or other discomfort occurs, seek medical attention promptly+- Quit smoking and limit alcohol, maintain good living habitsdiff --git a/tests/test_data/en/medicine/package_insert_sample.md b/tests/test_data/en/medicine/package_insert_sample.mdnew file mode 100644index 0000000..da5a6f9--- /dev/null+++ b/tests/test_data/en/medicine/package_insert_sample.md@@ -0,0 +1,80 @@+# Package Insert++## [Product Name]+Generic Name: Metformin Hydrochloride Sustained-release Tablets+Brand Name: Glucophage+English Name: Metformin Hydrochloride Sustained-release Tablets++## [Ingredients]+The main active ingredient of this product is metformin hydrochloride, with chemical name: 1,1-Dimethylbiguanide hydrochloride.+Molecular Formula: C4H11N5·HCl+Molecular Weight: 165.63++## [Description]+This product is a white or off-white film-coated tablet, appearing white after removing the coating.++## [Indications]+1. For type 2 diabetes patients whose blood glucose is not satisfactorily controlled by diet alone, especially those who are obese, may also help with weight reduction.+2. Effective for patients with poor response to certain sulfonylureas.+3. Can be combined with sulfonylureas or insulin to improve glycemic control.++## [Strength]+500mg (calculated as metformin hydrochloride)++## [Dosage and Administration]+Oral administration, taken with or after meals. Starting dose is usually once daily, 500mg, taken with dinner. Adjust dosage based on blood glucose and urine glucose, maximum daily dose should not exceed 2000mg. If satisfactory effect is not achieved with once daily 2000mg, may change to twice daily, 1000mg each time.++## [Adverse Reactions]+### 1. Common Adverse Reactions (Incidence >1%)+- **Gastrointestinal reactions**: Nausea, vomiting, diarrhea, abdominal pain, indigestion, heartburn+- **Systemic reactions**: Fatigue, headache, dizziness++### 2. Less Common Adverse Reactions (Incidence 0.1%-1%)+- **Skin reactions**: Rash, pruritus+- **Metabolic abnormalities**: Hypoglycemia (when combined with sulfonylureas or insulin)+- **Laboratory abnormalities**: Elevated lactate dehydrogenase, elevated ALT, elevated AST++### 3. Rare Adverse Reactions (Incidence <0.1%)+- **Lactic acidosis**: Manifested as fatigue, nausea, vomiting, abdominal pain, rapid and deep breathing, consciousness disturbance+- **Hematologic system**: Agranulocytosis, thrombocytopenia+- **Allergic reactions**: Urticaria, angioneurotic edema+- **Hepatobiliary abnormalities**: Hepatitis, cholestatic jaundice++### 4. Adverse Reaction Incidence Statistics+| System | Adverse Reaction | Incidence |+| :--- | :--- | :--- |+| **GI** | Nausea | 5.2% |+| **GI** | Vomiting | 2.6% |+| **GI** | Diarrhea | 4.5% |+| **GI** | Abdominal pain | 2.1% |+| **Systemic** | Fatigue | 1.8% |+| **Systemic** | Headache | 1.5% |+| **Skin** | Rash | 0.3% |+| **Metabolic** | Hypoglycemia | 0.2% (monotherapy), 1.5% (combined with sulfonylureas) |+| **Hepatobiliary** | Elevated transaminases | 0.1% |+| **Hematologic** | Agranulocytosis | <0.1% |++## [Contraindications]+1. **Absolute Contraindications**:+ - Patients allergic to metformin or other ingredients+ - Severe renal insufficiency (eGFR<30ml/min/1.73m²)+ - Metabolic acidosis (including diabetic ketoacidosis)+ - Severe infection, shock, or other diseases that may cause tissue hypoxia+ - Alcoholics+ - Patients with uncorrected vitamin B12 or folate deficiency+ - Within 48 hours before and after intravascular iodinated contrast agent administration++2. **Relative Contraindications**:+ - Renal insufficiency (eGFR 30-45ml/min/1.73m²)+ - Hepatic insufficiency+ - Heart failure+ - Severe pulmonary insufficiency+ - Pregnant and breastfeeding women++## [Precautions]+1. **Lactic Acidosis**: Metformin can cause lactic acidosis, which is rare but can be fatal. If it occurs, discontinue medication immediately and seek medical attention.+2. **Renal Function Monitoring**: Check renal function before and during medication, especially in elderly patients.+3. **Liver Function Monitoring**: Check liver function regularly during medication.+4. **Blood Glucose Monitoring**: Monitor blood glucose and glycated hemoglobin regularly during medication.+5. **Vitamin B12 Deficiency**: Long-term metformin use may reduce vitamin B12 absorption; regular monitoring of vitamin B12 levels is recommended.+6. **Around Surgery**: Discontinue metformin before elective surgery; may resume after renal function recovers postoperatively.diff --git a/tests/test_data/en/medicine/pathology_report_sample.md b/tests/test_data/en/medicine/pathology_report_sample.mdnew file mode 100644index 0000000..6945698--- /dev/null+++ b/tests/test_data/en/medicine/pathology_report_sample.md@@ -0,0 +1,80 @@+# Pathology Examination Report++## Basic Information+- **Pathology Number**: 2026-02001+- **Patient Name**: Li Si (Susan Lee)+- **Gender**: Female+- **Age**: 58 years old+- **Department**: Breast Surgery+- **Submission Date**: February 15, 2026+- **Report Date**: February 20, 2026++## Specimen Information+- **Specimen Type**: Right breast modified radical mastectomy specimen+- **Specimen Size**: 20cm×15cm×5cm+- **Tumor Location**: Upper outer quadrant of right breast+- **Tumor Size**: 3.5cm×2.5cm×2.0cm+- **Margin Status**: No cancer tissue found in superior, inferior, medial, lateral, or basal margins++## Histological Examination++### Gross Examination+- **Tumor Morphology**: Irregular shape, hard texture, unclear borders+- **Cut Surface Features**: Grayish-white, fish-flesh appearance, with necrosis+- **Surrounding Tissue**: Cancer infiltration visible in adipose tissue++### Microscopic Examination+- **Histological Type**: Invasive ductal carcinoma, no special type+- **Histological Grade**: Grade II (Bloom-Richardson grading)+- **Mitotic Figures**: 10/10HPF+- **Necrosis**: Focal necrosis visible+- **Vascular Invasion**: Lymphatic invasion present, no vascular invasion+- **Perineural Invasion**: No perineural invasion++## Immunohistochemistry Examination++### Routine Immunohistochemistry+- **ER**: (+, 90%)+- **PR**: (+, 70%)+- **HER2**: (2+)+- **Ki-67**: (+, 30%)+- **E-cadherin**: (+)+- **P120**: (membrane +)+- **CK5/6**: (-)+- **EGFR**: (-)++### FISH Testing+- **HER2 Gene Amplification**: Positive (HER2/CEP17 ratio = 3.2)++## Molecular Testing++### Genetic Testing+- **BRCA1**: Wild type+- **BRCA2**: Wild type+- **PIK3CA**: E545K mutation+- **TP53**: R273H mutation+- **ERBB2**: No mutation++## Lymph Node Examination+- **Axillary Lymph Nodes**: 20 nodes, 3 with cancer metastasis+- **Sentinel Lymph Nodes**: 2 nodes, 1 with cancer metastasis+- **Internal Mammary Lymph Nodes**: 0 nodes, no cancer metastasis++## TNM Staging+- **T Stage**: T2 (tumor maximum diameter >2cm and ≤5cm)+- **N Stage**: N1 (1-3 axillary lymph node metastases)+- **M Stage**: M0 (no distant metastasis)+- **Pathological Stage**: pT2N1M0, Stage IIB++## Other Examinations+- **Intracapsular Venous Tumor Thrombus**: Positive+- **Perineural Invasion**: Negative+- **Skin Invasion**: Negative+- **Pectoralis Fascia Invasion**: Negative++## Pathological Diagnosis+Right breast invasive ductal carcinoma, no special type, Grade II, pT2N1M0, Stage IIB. IHC: ER (+, 90%), PR (+, 70%), HER2 (2+, FISH positive), Ki-67 (+, 30%). Molecular testing: PIK3CA E545K mutation, TP53 R273H mutation, BRCA1/2 wild type. Axillary lymph node metastasis (3/20), sentinel lymph node metastasis (1/2).++## Remarks+1. Comprehensive judgment recommended in combination with clinical and other examination results+2. HER2-targeted therapy recommendeddiff --git a/tests/test_data/en/medicine/textbook_sample.md b/tests/test_data/en/medicine/textbook_sample.mdnew file mode 100644index 0000000..4be2b1c--- /dev/null+++ b/tests/test_data/en/medicine/textbook_sample.md@@ -0,0 +1,79 @@+# Internal Medicine (8th Edition)++## Chapter 1: Respiratory Diseases++### 1.1 Chronic Obstructive Pulmonary Disease++#### 1.1.1 Etiology and Pathogenesis+Chronic obstructive pulmonary disease (COPD) is a preventable and treatable disease characterized by persistent airflow limitation that is usually progressive and associated with an enhanced chronic inflammatory response in the airways and lung to noxious particles or gases.++**Risk Factors**:+- **Environmental factors**: Tobacco smoke, occupational dust and chemicals, air pollution, infection+- **Individual factors**: Genetic factors (such as alpha-1 antitrypsin deficiency), abnormal lung development, airway hyperresponsiveness++**Pathogenesis**:+1. Airway inflammation: Neutrophils, macrophages, T lymphocytes and other inflammatory cells participate+2. Oxidative stress: Increased oxidant production, decreased antioxidant capacity+3. Protease-antiprotease imbalance: Increased protease activity, decreased antiprotease activity+4. Airway remodeling: Structural changes in airways leading to airflow limitation++### 1.2 Bronchial Asthma++#### 1.2.1 Definition and Classification+Bronchial asthma is a heterogeneous disease characterized by chronic airway inflammation involving multiple cells (such as eosinophils, mast cells, T lymphocytes, neutrophils, airway epithelial cells) and cellular components.++**Classification**:+- Allergic asthma+- Non-allergic asthma+- Late-onset asthma+- Severe asthma++### 1.3 Pneumonia++#### 1.3.1 Classification+- By anatomy: Lobar pneumonia, lobular pneumonia, interstitial pneumonia+- By etiology: Bacterial pneumonia, viral pneumonia, mycoplasma pneumonia, fungal pneumonia+- By setting of acquisition: Community-acquired pneumonia, hospital-acquired pneumonia++## Chapter 2: Cardiovascular Diseases++### 2.1 Hypertension++#### 2.1.1 Definition and Classification+Hypertension is defined as systolic blood pressure ≥140 mmHg and/or diastolic blood pressure ≥90 mmHg on at least 3 separate occasions without antihypertensive medication.++**Blood Pressure Classification**:+- Normal blood pressure: SBP <120 mmHg and DBP <80 mmHg+- Normal high: SBP 120-139 mmHg and/or DBP 80-89 mmHg+- Hypertension:+ - Grade 1: SBP 140-159 mmHg and/or DBP 90-99 mmHg+ - Grade 2: SBP ≥160 mmHg and/or DBP ≥100 mmHg++### 2.2 Coronary Heart Disease++#### 2.2.1 Pathophysiology+Coronary atherosclerosis is the main cause of coronary heart disease. Its pathological changes include lipid deposition, fibrous tissue proliferation, plaque formation, ultimately leading to coronary artery lumen stenosis or obstruction, causing myocardial ischemia, hypoxia, or necrosis.++**Clinical Types**:+- Asymptomatic myocardial ischemia+- Angina pectoris+- Myocardial infarction+- Ischemic cardiomyopathy+- Sudden cardiac death++## Chapter 3: Digestive System Diseases++### 3.1 Peptic Ulcer Disease++#### 3.1.1 Etiology and Pathogenesis+Peptic ulcer disease mainly refers to chronic ulcers occurring in the stomach and duodenum, namely gastric ulcer (GU) and duodenal ulcer (DU).++**Main Causes**:+- Helicobacter pylori (Hp) infection+- Nonsteroidal anti-inflammatory drug (NSAIDs) use+- Gastric acid and pepsin+- Genetic factors+- Other factors: Smoking, alcohol, stress, etc.++**Pathogenesis**:+Imbalance between defensive factors (mucosal barrier, bicarbonate secretion, blood flow) and aggressive factors (acid, pepsin, Hp, NSAIDs) in the stomach and duodenum leads to peptic ulcer development.diff --git a/tests/test_data/en/tcm/herb_property_single.md b/tests/test_data/en/tcm/herb_property_single.mdnew file mode 100644index 0000000..ae39721--- /dev/null+++ b/tests/test_data/en/tcm/herb_property_single.md@@ -0,0 +1,18 @@+# Materia Medica Properties - Single Herb Sample++## Ginseng (Ren Shen)++### Name Explanation+Ginseng, also known as Huang Shen, Xue Shen, Ren Xian, Gui Gai, Shen Cao, Tu Jing, Di Jing, Hai Yu, Zhou Mian Huan Dan.++### Collected Explanations+"Bie Lu" states: Ginseng grows in the valleys of Shangdang and Liaodong. The roots are harvested in February, April, and early August, scraped with bamboo knife and dried in shade, must not be exposed to wind.++### Properties+Sweet, slightly cold, non-toxic.++### Main Indications+Tonifies the five zang organs, calms the spirit and consciousness, stabilizes the soul and ethereal soul, stops palpitations and fright, eliminates pathogenic factors, improves vision, opens the heart and benefits intelligence. Long-term use lightens the body and extends lifespan.++### Channel Entries+Enters the Spleen meridian, Lung meridian, Heart meridiandiff --git a/tests/test_data/en/tcm/herbal_compendium_sample.md b/tests/test_data/en/tcm/herbal_compendium_sample.mdnew file mode 100644index 0000000..c34cc6b--- /dev/null+++ b/tests/test_data/en/tcm/herbal_compendium_sample.md@@ -0,0 +1,55 @@+# Materia Medica Classics Sample++## Ginseng (Ren Shen)++### Name Explanation+Ginseng, also known as Huang Shen, Xue Shen, Ren Xian, Gui Gai, Shen Cao, Tu Jing, Di Jing, Hai Yu, Zhou Mian Huan Dan.++### Collected Explanations+"Bie Lu" states: Ginseng grows in the valleys of Shangdang and Liaodong. The roots are harvested in February, April, and early August, scraped with bamboo knife and dried in shade, must not be exposed to wind.++### Properties+Sweet, slightly cold, non-toxic.++### Main Indications+Tonifies the five zang organs, calms the spirit and consciousness, stabilizes the soul and ethereal soul, stops palpitations and fright, eliminates pathogenic factors, improves vision, opens the heart and benefits intelligence. Long-term use lightens the body and extends lifespan.++### Attached Formulas+1. Syndrome of yin collapse and yang exhaustion: Ginseng one liang, aconite one piece (processed), divided into four portions, decocted with water for oral administration.+2. Weak spleen and stomach: Ginseng, white atractylodes each two qian, poria, licorica each one qian, decocted with water for oral administration.++## Licorice (Gan Cao)++### Name Explanation+Licorice, also known as Mi Gan, Mi Cao, Mei Cao, Cao, Ling Tong, Guo Lao.++### Collected Explanations+"Bie Lu" states: Licorice grows in the valleys of western Henan and Jishan, and upper commandery. Roots are harvested on the second and eighth month on auspicious days, dried in sun, ready in ten days.++### Properties+Sweet, balanced, non-toxic.++### Main Indications+Cold and heat pathogens in the five zang and six fu organs, strengthens tendons and bones, promotes muscle growth, doubles qi and strength, treats金山 (unspecified) swelling, relieves toxicity. Long-term use lightens the body and extends lifespan.++### Attached Formulas+1. Sore throat in cold damage: Licorice two liang, platycodon one liang, decocted with water for oral administration.+2. Pediatric heat cough: Licorice two liang, pig gall juice soaked for five nights, roasted and ground into powder, made into honey pills the size of mung beans, ten pills taken after meals with thin mint soup.++## Astragalus (Huang Qi)++### Name Explanation+Astragalus, also known as Dai Shen, Dai Shen, Bai Ben, Wang Sun, Bai Yao Mian, Mian Huang Qi, Du Shen, Shu Zhi, Huang Qi.++### Collected Explanations+"Bie Lu" states: Astragalus grows in the valleys of Shu commandery, Baishui, and Hanzhong. Roots are harvested in February and October, dried in shade.++### Properties+Sweet, slightly warm, non-toxic.++### Main Indications+Long-term ulcers and decayed sores, expels pus and relieves pain, treats severe wind and leprosy, five hemorrhoids and mouse fistula, tonifies deficiency, pediatric hundred diseases.++### Attached Formulas+1. Qi deficiency with night sweats: Astragalus two liang, white atractylodes one liang, fangfeng half liang, ground into powder, three qian each time, decocted with water for oral administration.+2. Qi deficiency with fatigue: Astragalus, ginseng each one liang, decocted with water for oral administration.diff --git a/tests/test_data/en/tcm/medical_case_record_sample.md b/tests/test_data/en/tcm/medical_case_record_sample.mdnew file mode 100644index 0000000..8058bb6--- /dev/null+++ b/tests/test_data/en/tcm/medical_case_record_sample.md@@ -0,0 +1,61 @@+# Famous Physician Case Records Sample++## Taiyang Wind-Strike Case++### Symptoms+Fever, sweating, aversion to wind, moderate pulse, nasal wheezing, dry retching.++### Pattern Differentiation+Taiyang wind-strike: the yang is floating and the yin is weak. Yang floating generates heat; yin weakness causes spontaneous sweating. Gradual aversion to cold, slight aversion to wind, mild fever, nasal wheezing and dry retching indicate that Guizhi Decoction should be the primary treatment.++### Treatment Method+Releases the muscles and expels pathogens, harmonizes the Ying and Wei.++### Formula+Guizhi Decoction (Cinnamon Twig Decoction): Cinnamon twig three liang (removed bark), peony three liang, licorica two liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken).++### Preparation and Administration+Grind five ingredients, decoct with seven sheng of water over gentle heat until three sheng remains. Remove the residue, let cool to appropriate temperature, take one sheng. After taking, drink one sheng plus of hot thin porridge to assist the medicinal force. Cover with clothing for approximately one hour until the whole body has slight sweating is best. Do not cause heavy sweating like water flowing, or the disease will not be eliminated.++### Case Analysis+This case is a typical Taiyang wind-strike pattern. The patient was affected by wind pathogen, defensive qi is insecure, and ying yin is leaking outward, hence symptoms of fever, sweating, and aversion to wind. Treatment with Guizhi Decoction releases the muscles and expels pathogens, harmonizes ying and wei, removing the pathogen and restoring proper qi.++## Taiyang Cold-Damage Case++### Symptoms+Headache, fever, body pain, lower back pain, joint pain, aversion to wind, no sweating with wheezing.++### Pattern Differentiation+In Taiyang disease with headache, fever, body pain, lower back pain, joint pain, aversion to wind, no sweating with wheezing, Mahuang Decoction should be the primary treatment.++### Treatment Method+Promotes sweating and releases the exterior, diffuses the lung to relieve asthma.++### Formula+Mahuang Decoction (Ephedra Decoction): Ephedra three liang (removed nodes), cinnamon twig two liang (removed bark), licorica one liang (honey-fried), apricot kernel seventy pieces (removed skin and tip).++### Preparation and Administration+Decoct four ingredients with nine sheng of water. First decoct mahuang, reduce by two sheng, remove the foam, add the other herbs, decoct until two and a half sheng remains. Remove the residue, take eight ge warm. Cover to induce slight sweating. No need for hot porridge. Follow the same guidelines as Guizhi Decoction.++### Case Analysis+This case is a typical Taiyang cold-damage pattern. The patient was affected by cold pathogen, defensive yang is restrained, and ying yin is stagnated, hence symptoms of headache, fever, body pain, and no sweating. Treatment with Mahuang Decoction promotes sweating and releases the exterior, diffuses the lung to relieve asthma, causing the cold pathogen to resolve through sweating.++## Shaoyang Disease Case++### Symptoms+Alternating chills and fever, chest and hypochondriac fullness, dispirited and reluctant to eat, irritability, frequent vomiting, bitter taste, dry throat, dizziness.++### Pattern Differentiation+When wind-cold damage occurs for five to six days with alternating chills and fever, chest and hypochondriac fullness, dispirited and reluctant to eat, irritability and frequent vomiting, or chest discomfort without vomiting, or thirst, or abdominal pain, or hypochondriac hardness, or palpitations below the heart and urinary difficulty, or no thirst with slight fever externally, or cough—Xiaochaihu Decoction should be the primary treatment.++### Treatment Method+Harmonizes Shaoyang.++### Formula+Xiaochaihu Decoction (Minor Bupleurum Decoction): Bupleurum half jin, scutellaria three liang, ginseng three liang, licorica three liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken), pinellia half sheng (washed).++### Preparation and Administration+Decoct seven ingredients with one dou and two sheng of water until six sheng remains. Remove the residue, decoct again until three sheng remains. Take one sheng warm, three times daily.++### Case Analysis+This case is a typical Shaoyang disease pattern. The patient was affected by pathogen invading Shaoyang, the pivot mechanism is unbalanced, hence symptoms of alternating chills and fever, chest and hypochondriac fullness, irritability, and frequent vomiting. Treatment with Xiaochaihu Decoction harmonizes Shaoyang, removing the pathogen and curing the disease.diff --git a/tests/test_data/en/tcm/meridian_treatise_sample.md b/tests/test_data/en/tcm/meridian_treatise_sample.mdnew file mode 100644index 0000000..7874319--- /dev/null+++ b/tests/test_data/en/tcm/meridian_treatise_sample.md@@ -0,0 +1,53 @@+# Meridian and Acupoint Treatise Sample++## Twelve Primary Channels and Their Pathways++### Hand Taiyin Lung Channel++Begins in the middle jiao, descends to connect with the large intestine, returns to loop around the stomach opening, passes through the diaphragm, and belongs to the lung. From the lung system, it emerges laterally at the axilla, descends along the inner aspect of the upper arm, travels anterior to the hand Shaoyin and hand Jueyin channels, reaches the inner aspect of the elbow, ascends along the radial border of the upper aspect of the forearm, enters the cun Kou (radial pulse), ascends to the thenar eminence, follows the thenar border, and emerges at the tip of the thumb. Its branch: from the area posterior to the wrist, emerges directly at the medial border of the index finger, and ends at its tip.++### Hand Yangming Large Intestine Channel++Begins at the tip of the index finger, follows the superior border of the index finger, emerges between the two metacarpal bones at Hegu, ascends between the two tendons on the radial side of the wrist, travels along the superior border of the forearm, enters the lateral aspect of the elbow, ascends along the lateral anterior aspect of the upper arm, reaches the shoulder, emerges at the anterior border of the shoulder socket, ascends to exit at the meeting point of the spine and shoulder, descends to enter the supraclavicular fossa, connects with the lung, passes through the diaphragm, belongs to the large intestine. Its branch: from the supraclavicular fossa, ascends the neck, penetrates the cheek, enters the lower teeth, returns and circles the lips, crosses at Renzhong, with the left going to the right and right to the left, ascends beside the nose to the lateral side of the nose.++### Foot Yangming Stomach Channel++Begins at the side of the nose, meets the foot Taiyang channel at Yingtang, descends along the lateral aspect of the nose, enters the upper teeth, returns and circles the lips, descends to Chengjiang, returns along the inferior border of the jaw, emerges at Dagying, follows Jiache, ascends in front of the ear, passes through Kukezhuren, follows the hairline to the forehead. Its branch: from Dagying anteriorly, descends to Renying, follows the throat, enters the supraclavicular fossa, descends through the diaphragm, belongs to the stomach, connects with the spleen. Its direct branch: from the supraclavicular fossa, descends along the inner aspect of the breast, descends beside the umbilicus, enters Qijie, the Stomach Qi Street. Its branch: originates from the stomach opening, descends along the interior of the abdomen, reaches Qijie where it joins the main channel. From there it descends to Biguan, reaches Futu, descends through the knee cup, follows the lateral border of the tibia, descends to the dorsum of the foot, enters the middle toe. Its branch: descends three cun below the knee, and enters the lateral side of the middle toe. Its branch: separates at the dorsum of the foot, enters the great toe, emerges at its tip.++### Foot Taiyin Spleen Channel++Begins at the tip of the great toe, follows the medial border of the great toe, passes posterior to the medial malleolus, ascends along the anterior border of the medial malleolus, ascends along the posterior aspect of the tibia, crosses anterior to the foot Shaoyin channel, ascends along the anterior border of the thigh, enters the abdomen, belongs to the spleen, connects with the stomach, passes through the diaphragm, attaches to the throat, connects with the tongue root, spreads under the tongue. Its branch: again separates from the stomach, ascends through the diaphragm, and enters the heart.++### Hand Shaoyin Heart Channel++Begins in the heart, exits to belong to the heart connecting system, descends through the diaphragm, connects with the small intestine. Its branch: from the heart connecting system, ascends alongside the throat, connects with the eye connecting system. Its direct branch: returns from the heart connecting system, ascends to the lung, descends to emerge at the axilla, descends along the posterior border of the inner aspect of the upper arm, travels posterior to the hand Taiyin and hand Jueyin channels, reaches the inner aspect of the elbow, follows the posterior border of the inner aspect of the forearm, reaches the tip of the styloid process of the ulna, enters the palm on the posterior border of the ulnar side, follows the inner side of the little finger, and emerges at its tip.++### Hand Taiyang Small Intestine Channel++Begins at the tip of the little finger, follows the lateral border of the hand, ascends at the wrist, emerges between the ulna and radius, ascends along the inferior border of the radius, emerges between the two tendons on the lateral side of the elbow, ascends along the posterior border of the lateral aspect of the upper arm, exits at the shoulder joint, circles the scapula, crosses at the shoulder to meet the channel, enters the supraclavicular fossa, connects with the heart, follows the throat through the diaphragm, reaches the stomach, belongs to the small intestine. Its branch: from the supraclavicular fossa, follows the neck, ascends the cheek, reaches the lateral canthus, returns and enters the ear. Its branch: separates at the cheek, ascends to the nose, reaches the medial canthus, diagonally connects to the zygomatic region.++## Commonly Used Acupoints++### Hegu (LI4)++Location: On the dorsum of the hand, between the 1st and 2nd metacarpal bones, at the midpoint of the 2nd metacarpal bone's radial side.++Indications: Headache, red swollen painful eyes, nosebleed, toothache, lockjaw, facial paralysis, deafness, mumps, sore throat, fever without sweating, excessive sweating, abdominal pain, constipation, amenorrhea, difficult labor.++### Zusanli (ST36)++Location: On the anterolateral aspect of the lower leg, 3 cun below the Dubi, one finger-breadth from the anterior border of the tibia.++Indications: Stomach pain, vomiting, dysphagia, abdominal distension, diarrhea, dysentery, constipation, breast abscess, intestinal abscess, lower limb pain and numbness, edema, manic-depressive disorder, beriberi, emaciation due to deficiency.++### Sanyinjiao (SP6)++Location: On the medial aspect of the lower leg, 3 cun above the tip of the medial malleolus, posterior to the medial border of the tibia.++Indications: Intestinal rumbling, abdominal distension, diarrhea, menstrual disorders, leukorrhea, vaginal prolapse, infertility, difficult labor, seminal emission, impotence, enuresis, hernia, insomnia, lower limb flaccidity and numbness, beriberi.++### Guanyuan (CV4)++Location: On the lower abdomen, on the anterior midline, 3 cun below the center of the umbilicus.++Indications: Stroke collapse syndrome, deficiency exhaustion, emaciation and weakness, lower abdominal pain, cholera vomiting and diarrhea, dysentery, rectal prolapse, hernia, bloody stools, bloody urine, dysuria, frequent urination, urinary retention, seminal emission, turbid white leukorrhea, impotence, premature ejaculation, menstrual disorders, amenorrhea, dysmenorrhea, vaginal prolapse, flooding and spotting, vulvar itching, lochia, retained placenta, wasting thirst, dizziness.diff --git a/tests/test_data/en/tcm/prescription_formulary_sample.md b/tests/test_data/en/tcm/prescription_formulary_sample.mdnew file mode 100644index 0000000..5e30836--- /dev/null+++ b/tests/test_data/en/tcm/prescription_formulary_sample.md@@ -0,0 +1,52 @@+# Classical Formula Sample++## Guizhi Tang (Cinnamon Twig Decoction)++### Composition+Cinnamon twig three liang (removed bark), peony three liang, licorica two liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken).++### Preparation and Administration+Grind five ingredients, decoct with seven sheng of water over gentle heat until three sheng remains. Remove the residue, let cool to appropriate temperature, take one sheng. After taking, drink one sheng plus of hot thin porridge to assist the medicinal force. Cover with clothing for approximately one hour until the whole body has slight sweating is best. Do not cause heavy sweating like water flowing, or the disease will not be eliminated. If sweating occurs and the disease is cured, stop taking the remaining portion. If sweating does not occur, take again following the same method. If still no sweating, take again shortening the interval, approximately three portions within half a day. For severe disease, take continuously day and night, observe throughout; if after taking one portion the disease remains, take another portion; if sweating does not occur, may take up to two or three portions.++### Functions and Indications+Releases the muscles and expels pathogens, harmonizes the Ying and Wei. Main treatment of Taiyang wind-strike: yang is floating and yin is weak, yang floating generates heat, yin weakness causes spontaneous sweating. Patient presents with slight aversion to cold, slight aversion to wind, mild fever, nasal wheezing and dry retching.++### Jun-Chen-Zuo-Shi (Monarch-Minister-Assistant-Emissary) Analysis+- **Monarch (Jun)**: Cinnamon twig, acrid and warm, releases the muscles and expels pathogens, warms and unblocks the channels.+- **Minister (Chen)**: Peony, sour and bitter, slightly cold, nourishes yin and constricts ying.+- **Assistant (Zuo)**: Fresh ginger, acrid and warm, assists cinnamon twig in releasing the exterior; jujube, sweet and warm, tonifies qi and supplements the middle.+- **Emissary (Shi)**: Licorica, sweet and warm, harmonizes all the medicinals.++## Mahuang Tang (Ephedra Decoction)++### Composition+Ephedra three liang (removed nodes), cinnamon twig two liang (removed bark), licorica one liang (honey-fried), apricot kernel seventy pieces (removed skin and tip).++### Preparation and Administration+Decoct four ingredients with nine sheng of water. First decoct mahuang, reduce by two sheng, remove the foam, add the other herbs, decoct until two and a half sheng remains. Remove the residue, take eight ge warm. Cover to induce slight sweating. No need for hot porridge. Follow the same guidelines as Guizhi Decoction.++### Functions and Indications+Promotes sweating and releases the exterior, diffuses the lung to relieve asthma. Main treatment of Taiyang disease: headache, fever, body pain, lower back pain, joint pain, aversion to wind, no sweating with wheezing.++### Jun-Chen-Zuo-Shi Analysis+- **Monarch (Jun)**: Ephedra, acrid and warm, promotes sweating and releases the exterior, diffuses the lung to relieve asthma.+- **Minister (Chen)**: Cinnamon twig, acrid and warm, releases the muscles and expels pathogens, warms and unblocks the channels.+- **Assistant (Zuo)**: Apricot kernel, bitter and warm, descends and directs lung qi downward, relieves asthma and stops coughing.+- **Emissary (Shi)**: Licorica, sweet and warm, harmonizes all the medicinals.++## Xiaochaihu Tang (Minor Bupleurum Decoction)++### Composition+Bupleurum half jin, scutellaria three liang, ginseng three liang, licorica three liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken), pinellia half sheng (washed).++### Preparation and Administration+Decoct seven ingredients with one dou and two sheng of water until six sheng remains. Remove the residue, decoct again until three sheng remains. Take one sheng warm, three times daily.++### Functions and Indications+Harmonizes Shaoyang. Main treatment: when wind-cold damage occurs for five to six days with alternating chills and fever, chest and hypochondriac fullness, dispirited and reluctant to eat, irritability and frequent vomiting; or chest discomfort without vomiting; or thirst; or abdominal pain; or hypochondriac hardness; or palpitations below the heart and urinary difficulty; or no thirst with slight fever externally; or cough.++### Jun-Chen-Zuo-Shi Analysis+- **Monarch (Jun)**: Bupleurum, bitter and neutral, harmonizes Shaoyang, courses the liver and gallbladder.+- **Minister (Chen)**: Scutellaria, bitter and cold, clears and drains Shaoyang heat.+- **Assistant (Zuo)**: Ginseng and jujube, tonify qi and fortify the spleen; pinellia and fresh ginger, harmonize the stomach and direct rebellion downward.+- **Emissary (Shi)**: Licorica, harmonizes all the medicinals.diff --git a/tests/test_data/en/tcm/prescription_modification_sample.md b/tests/test_data/en/tcm/prescription_modification_sample.mdnew file mode 100644index 0000000..c8059e5--- /dev/null+++ b/tests/test_data/en/tcm/prescription_modification_sample.md@@ -0,0 +1,59 @@+# Formula Modification Logic Sample++## Guizhi Tang (Cinnamon Twig Decoction) Modifications++### Original Formula+Guizhi Tang: Cinnamon twig three liang (removed bark), peony three liang, licorica two liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken).++### Modification 1: Accompanying Neck and Back Stiffness+If the disease is accompanied by neck and back stiffness, **add Pueraria four liang**.++### Modification 2: Accompanying Asthma+If the disease is accompanied by asthma, **add Magnolia bark two liang, apricot kernel fifty pieces**.++### Modification 3: Accompanying Abdominal Fullness and Pain+If the disease is accompanied by abdominal fullness and pain, **remove peony, add Aconite one piece**.++### Modification 4: Accompanying Yang Deficiency with Sweating+If the disease is accompanied by yang deficiency with sweating, **remove peony, add Aconite one piece**.++---++## Mahuang Tang (Ephedra Decoction) Modifications++### Original Formula+Mahuang Tang: Ephedra three liang (removed nodes), cinnamon twig two liang (removed bark), licorica one liang (honey-fried), apricot kernel seventy pieces (removed skin and tip).++### Modification 1: Accompanying Internal Heat+If the disease is accompanied by internal heat, **remove cinnamon twig, add Gypsum half jin**.++### Modification 2: Accompanying Dampness+If the disease is accompanied by dampness, **add Atractylodes two liang**.++---++## Xiaochaihu Tang (Minor Bupleurum Decoction) Modifications++### Original Formula+Xiaochaihu Tang: Bupleurum half jin, scutellaria three liang, ginseng three liang, licorica three liang (honey-fried), fresh ginger three liang (sliced), jujube twelve pieces (broken), pinellia half sheng (washed).++### Modification 1: Chest Restlessness without Vomiting+If the disease is accompanied by chest restlessness without vomiting, **remove Pinellia and Ginseng, add Trichosanthes fruit one piece**.++### Modification 2: Accompanying Thirst+If the disease is accompanied by thirst, **remove Pinellia, add Ginseng to total four and a half liang, add Trichosanthes root four liang**.++### Modification 3: Accompanying Abdominal Pain+If the disease is accompanied by abdominal pain, **remove Scutellaria, add Peony three liang**.++### Modification 4: Accompanying Hypochondriac Hardness and Stuffiness+If the disease is accompanied by hypochondriac hardness and stuffiness, **remove Jujube, add Ostrea four liang**.++### Modification 5: Accompanying Palpitations below the Heart+If the disease is accompanied by palpitations below the heart, **remove Scutellaria, add Poria four liang**.++### Modification 6: No Thirst with Mild External Heat+If the disease is accompanied by no thirst with mild external heat, **remove Ginseng, add Cinnamon twig three liang, cover to induce slight sweating and cure**.++### Modification 7: Accompanying Cough+If the disease is accompanied by cough, **remove Ginseng, Jujube, and Fresh ginger, add Schisandra half sheng, dried ginger two liang**.diff --git a/tests/types/__init__.py b/tests/types/__init__.pyindex ee4509f..f507f4a 100644--- a/tests/types/__init__.py+++ b/tests/types/__init__.py@@ -1 +1 @@-"""Test suite for Hyper-Extract types module."""+"""Tests for hyperextract types module."""diff --git a/tests/types/test_graph.py b/tests/types/test_graph.pydeleted file mode 100644index 63aa5f3..0000000--- a/tests/types/test_graph.py+++ /dev/null@@ -1,271 +0,0 @@-"""Tests for AutoGraph - basic graph extraction pattern."""--import pytest-from pydantic import BaseModel, Field-from typing import Tuple-from ontomem.merger import MergeStrategy-from hyperextract.types.graph import AutoGraph---class Entity(BaseModel):- """Test schema for graph nodes."""-- name: str = Field(default="", description="Entity name")- entity_type: str = Field(default="unknown", description="Type of entity")---class Relation(BaseModel):- """Test schema for graph edges."""-- source: str = Field(default="", description="Source entity name")- target: str = Field(default="", description="Target entity name")- relation_type: str = Field(default="", description="Type of relation")---class TestAutoGraphBasics:- """Test basic AutoGraph initialization."""-- def test_initialization_one_stage(self, llm_client, embedder):- """Test AutoGraph initialization in one-stage mode."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- extraction_mode="one_stage",- )-- assert kg.extraction_mode == "one_stage"- assert kg.node_schema == Entity- assert kg.edge_schema == Relation- assert kg.empty() is True-- def test_initialization_two_stage(self, llm_client, embedder):- """Test AutoGraph initialization in two-stage mode."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- assert kg.extraction_mode == "two_stage"- assert kg.empty() is True-- def test_custom_prompts(self, llm_client, embedder):- """Test that custom prompts are stored."""- custom_prompt = "Custom extraction prompt"- node_prompt = "Custom node prompt"- edge_prompt = "Custom edge prompt"-- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- prompt=custom_prompt,- prompt_for_node_extraction=node_prompt,- prompt_for_edge_extraction=edge_prompt,- )-- assert custom_prompt in kg.prompt- assert node_prompt in kg.node_prompt- assert edge_prompt in kg.edge_prompt---class TestAutoGraphExtraction:- """Test graph extraction operations."""-- def test_feed_text_one_stage(self, llm_client, embedder):- """Test feeding text in one-stage mode."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- extraction_mode="one_stage",- )-- kg.feed_text("Alice knows Bob. Bob knows Charlie.")-- assert not kg.empty()- assert len(kg.data.nodes) > 0-- def test_feed_text_two_stage(self, llm_client, embedder):- """Test feeding text in two-stage mode."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- kg.feed_text("Alice is an engineer. Bob is a designer. They work together.")-- assert not kg.empty()- assert hasattr(kg.data, "nodes")- assert hasattr(kg.data, "edges")-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test extract() creates new instance."""- kg1 = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- kg2 = kg1.parse("""CEO Sarah leads Engineering where Bob manages Backend Systems.- Bob oversees both Database Architecture (led by Alice) and API Development (led by Tom).- Alice collaborates with Tom on performance optimization.""")-- assert kg1 is not kg2- assert kg1.empty() is True- # kg2 may or may not be empty depending on extraction results---class TestAutoGraphProperties:- """Test graph data properties."""-- def test_nodes_property(self, llm_client, embedder):- """Test accessing nodes from graph."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- kg.feed_text("""- Engineering Director Michael manages Sarah (Backend Lead) and James (Frontend Lead).- Sarah collaborates with DevOps engineer David on infrastructure.- James works with UI designer Lisa on frontend components.- Chief Architect Elena mentors the entire team on best practices.- """)-- nodes = kg.data.nodes- assert isinstance(nodes, list)- assert all(isinstance(n, Entity) for n in nodes)-- def test_edges_property(self, llm_client, embedder):- """Test accessing edges from graph."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- kg.feed_text("""- CEO John directs CTO Lisa and CFO David.- Lisa manages the core engineering team and reports to John.- David oversees financial planning and coordinates with John on budgets.- Senior Architect Elena advises Lisa on technical decisions.- """)-- edges = kg.data.edges- assert isinstance(edges, list)- assert all(isinstance(e, Relation) for e in edges)---class TestAutoGraphDeduplication:- """Test graph deduplication logic."""-- def test_node_deduplication_by_key(self, llm_client, embedder):- """Test that nodes with same key are deduplicated."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- # Manually create nodes- e1 = Entity(name="Alice", entity_type="person")- e2 = Entity(name="Alice", entity_type="person") # Same key- e3 = Entity(name="Bob", entity_type="person")-- # Add them manually to test deduplication- kg._node_memory.add([e1, e2, e3])-- # After deduplication via key, should have 2 nodes- assert kg._node_memory.size == 2-- def test_edge_deduplication_by_key(self, llm_client, embedder):- """Test that edges with same key are deduplicated."""- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- r1 = Relation(source="Alice", target="Bob", relation_type="knows")- r2 = Relation(source="Alice", target="Bob", relation_type="knows") # Same key- r3 = Relation(source="Bob", target="Charlie", relation_type="knows")-- # Add them manually to test deduplication- kg._edge_memory.add([r1, r2, r3])-- # After deduplication via key, should have 2 edges- assert kg._edge_memory.size == 2---class TestAutoGraphSchemaValidation:- """Test schema validation."""-- def test_edge_key_extractor_receives_relation_object(self, llm_client, embedder):- """Test that edge_key_extractor receives Relation objects."""- called_with = []-- def capture_edge_key(rel: Relation):- called_with.append(rel)- return f"{rel.source}-{rel.relation_type}-{rel.target}"-- kg = AutoGraph(- node_schema=Entity,- edge_schema=Relation,- node_key_extractor=lambda x: x.name,- edge_key_extractor=capture_edge_key,- nodes_in_edge_extractor=lambda x: (x.source, x.target),- llm_client=llm_client,- embedder=embedder,- )-- rel = Relation(source="A", target="B", relation_type="test")- capture_edge_key(rel)-- assert len(called_with) > 0- assert called_with[-1].source == "A"diff --git a/tests/types/test_graph_dangling.py b/tests/types/test_graph_dangling.pynew file mode 100644index 0000000..164af64--- /dev/null+++ b/tests/types/test_graph_dangling.py@@ -0,0 +1,74 @@+"""Unit tests for AutoGraph dangling edge pruning."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional, List++from hyperextract.types import AutoGraph+++class Entity(BaseModel):+ """Simple entity schema for testing."""++ name: str+ type: str+ properties: dict = Field(default_factory=dict)+++class Relation(BaseModel):+ """Simple relation schema for testing."""++ source: str+ target: str+ relation_type: str+++class TestAutoGraphDangling:+ """Test cases for AutoGraph dangling edge handling."""++ def test_graph_with_dangling_edges(self, llm_client, embedder):+ """Test that graph can contain dangling edges (no automatic pruning)."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])+ graph._edge_memory.add([+ Relation(source="Apple", target="Steve Jobs", relation_type="founded_by"),+ ])++ graph.build_index()++ nodes, edges = graph.search("Apple", top_k_nodes=2, top_k_edges=2)++ assert len(nodes) >= 1+++class TestAutoGraphMerge:+ """Test cases for AutoGraph merge functionality."""++ def test_merge_batch_data_single_graph(self, llm_client, embedder):+ """Test merging single graph."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])++ assert len(graph.nodes) == 1diff --git a/tests/types/test_graph_extraction.py b/tests/types/test_graph_extraction.pynew file mode 100644index 0000000..b7eb544--- /dev/null+++ b/tests/types/test_graph_extraction.py@@ -0,0 +1,188 @@+"""Unit tests for AutoGraph extraction functionality."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional, List++from hyperextract.types import AutoGraph+++class Entity(BaseModel):+ """Simple entity schema for testing."""++ name: str+ type: str+ properties: dict = Field(default_factory=dict)+++class Relation(BaseModel):+ """Simple relation schema for testing."""++ source: str+ target: str+ relation_type: str+++class TestAutoGraphExtraction:+ """Test cases for AutoGraph extraction functionality."""++ def test_init_with_schemas(self, llm_client, embedder):+ """Test initialization with node and edge schemas."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert graph.empty() is True+ assert len(graph.nodes) == 0+ assert len(graph.edges) == 0++ def test_node_schema_property(self, llm_client, embedder):+ """Test that node_schema property returns correct schema."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert graph.node_schema == Entity++ def test_edge_schema_property(self, llm_client, embedder):+ """Test that edge_schema property returns correct schema."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert graph.edge_schema == Relation++ def test_feed_text_adds_nodes_directly(self, llm_client, embedder):+ """Test that we can add nodes directly to memory without LLM call."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ Entity(name="Steve Jobs", type="PERSON", properties={}),+ ])++ assert len(graph._node_memory.items) == 2+ assert graph.empty() is False++ def test_feed_text_returns_self(self, llm_client, embedder):+ """Test that adding data returns self for method chaining."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([Entity(name="Apple", type="ORGANIZATION", properties={})])++ assert graph.empty() is False++ def test_clear_resets_state(self, llm_client, embedder):+ """Test that clear resets the graph to empty state."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([Entity(name="Apple", type="ORGANIZATION", properties={})])++ graph.clear()++ assert graph.empty() is True+ assert len(graph._node_memory.items) == 0++ def test_two_stage_extraction_mode(self, llm_client, embedder):+ """Test two-stage extraction mode initialization."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ extraction_mode="two_stage",+ )++ assert graph.extraction_mode == "two_stage"+++class TestAutoGraphInternal:+ """Test cases for AutoGraph internal state management."""++ def test_nodes_property(self, llm_client, embedder):+ """Test nodes property."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert isinstance(graph.nodes, list)++ def test_edges_property(self, llm_client, embedder):+ """Test edges property."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert isinstance(graph.edges, list)++ def test_metadata_initialized(self, llm_client, embedder):+ """Test that metadata is initialized correctly."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ assert "created_at" in graph.metadata+ assert "updated_at" in graph.metadatadiff --git a/tests/types/test_graph_search.py b/tests/types/test_graph_search.pynew file mode 100644index 0000000..c997549--- /dev/null+++ b/tests/types/test_graph_search.py@@ -0,0 +1,188 @@+"""Unit tests for AutoGraph search functionality."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional, List++from hyperextract.types import AutoGraph+++class Entity(BaseModel):+ """Simple entity schema for testing."""++ name: str+ type: str+ properties: dict = Field(default_factory=dict)+++class Relation(BaseModel):+ """Simple relation schema for testing."""++ source: str+ target: str+ relation_type: str+++class TestAutoGraphSearch:+ """Test cases for AutoGraph search functionality."""++ def test_build_index_nodes_only(self, llm_client, embedder):+ """Test building index for nodes only."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ Entity(name="Steve Jobs", type="PERSON", properties={}),+ ])++ graph.build_index(index_nodes=True, index_edges=False)++ assert graph._node_memory.has_index()++ def test_build_index_both(self, llm_client, embedder):+ """Test building index for both nodes and edges."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])+ graph._edge_memory.add([+ Relation(source="Apple", target="Steve Jobs", relation_type="founded_by"),+ ])++ graph.build_index(index_nodes=True, index_edges=True)++ assert graph._node_memory.has_index()+ assert graph._edge_memory.has_index()++ def test_build_node_index(self, llm_client, embedder):+ """Test building node index specifically."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])++ graph.build_node_index()++ assert graph._node_memory.has_index()++ def test_search_nodes(self, llm_client, embedder):+ """Test searching nodes."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ Entity(name="Steve Jobs", type="PERSON", properties={}),+ ])+ graph.build_node_index()++ results = graph.search_nodes("technology company", top_k=2)++ assert isinstance(results, list)++ def test_search_edges(self, llm_client, embedder):+ """Test searching edges."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._edge_memory.add([+ Relation(source="Apple", target="Steve Jobs", relation_type="founded_by"),+ Relation(source="Apple", target="Tim Cook", relation_type="led_by"),+ ])+ graph.build_edge_index()++ results = graph.search_edges("founder", top_k=2)++ assert isinstance(results, list)++ def test_search_unified(self, llm_client, embedder):+ """Test unified search interface."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])+ graph._edge_memory.add([+ Relation(source="Apple", target="Steve Jobs", relation_type="founded_by"),+ ])++ graph.build_index()++ nodes, edges = graph.search("Apple company", top_k_nodes=1, top_k_edges=1)++ assert isinstance(nodes, list)+ assert isinstance(edges, list)++ def test_search_with_top_k(self, llm_client, embedder):+ """Test search with top_k parameter."""+ graph = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )++ graph._node_memory.add([+ Entity(name="Apple", type="ORGANIZATION", properties={}),+ ])+ graph._edge_memory.add([+ Relation(source="Apple", target="Steve Jobs", relation_type="founded_by"),+ ])++ graph.build_index()++ nodes, edges = graph.search("company", top_k=2)++ assert isinstance(nodes, list)+ assert isinstance(edges, list)diff --git a/tests/types/test_hypergraph.py b/tests/types/test_hypergraph.pydeleted file mode 100644index bc8e795..0000000--- a/tests/types/test_hypergraph.py+++ /dev/null@@ -1,269 +0,0 @@-"""Tests for AutoHypergraph - hypergraph extraction pattern."""--import pytest-from pydantic import BaseModel, Field-from typing import List, Tuple-from ontomem.merger import MergeStrategy-from hyperextract.types.hypergraph import AutoHypergraph---class Person(BaseModel):- """Test schema for hypergraph nodes."""-- name: str = Field(default="", description="Person's name")---class Event(BaseModel):- """Test schema for hyperedges."""-- event_type: str = Field(default="", description="Type of event")- participants: List[str] = Field(default_factory=list, description="Names of participants")---class TestAutoHypergraphBasics:- """Test basic AutoHypergraph functionality."""-- def test_initialization(self, llm_client, embedder):- """Test AutoHypergraph initialization."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- assert hg.node_schema == Person- assert hg.edge_schema == Event- assert hg.extraction_mode == "two_stage"- assert hg.empty() is True-- def test_default_extraction_mode_is_two_stage(self, llm_client, embedder):- """Test that default extraction mode is two_stage."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- assert hg.extraction_mode == "two_stage"---class TestAutoHypergraphKeyGeneration:- """Test key generation and deduplication logic."""-- def test_edge_key_with_sorted_participants(self, llm_client, embedder):- """Test that edge keys are the same for same participants regardless of order."""- # Define key extractor with sorting- def edge_key(event: Event) -> str:- sorted_participants = "-".join(sorted(event.participants))- return f"{event.event_type}_{sorted_participants}"-- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=edge_key,- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- # Two events with same type and participants but different order- e1 = Event(event_type="meeting", participants=["Alice", "Bob", "Charlie"])- e2 = Event(event_type="meeting", participants=["Charlie", "Alice", "Bob"])-- key1 = edge_key(e1)- key2 = edge_key(e2)-- assert key1 == key2- assert key1 == "meeting_Alice-Bob-Charlie"-- def test_hyperedge_participants_extractor(self, llm_client, embedder):- """Test that nodes_in_edge_extractor returns all participants."""- def participants_extractor(event: Event) -> Tuple[str, ...]:- return tuple(event.participants)-- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=participants_extractor,- llm_client=llm_client,- embedder=embedder,- )-- event = Event(event_type="party", participants=["Alice", "Bob", "Dave"])- participants = participants_extractor(event)-- assert len(participants) == 3- assert "Alice" in participants- assert "Bob" in participants- assert "Dave" in participants---class TestAutoHypergraphExtraction:- """Test hypergraph extraction."""-- def test_feed_text_populates_hypergraph(self, llm_client, embedder):- """Test feeding text extracts nodes and hyperedges."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- hg.feed_text("Alice, Bob, and Charlie had a meeting. They collaborated on a project.")-- assert not hg.empty()- assert len(hg.data.nodes) > 0-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test extract() creates new instance."""- hg1 = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- hg2 = hg1.parse("""Annual planning conference brings together CEO, CFO, CTO,- VP Marketing, and VP Sales to discuss Q2 revenue targets and product roadmap priorities.""")-- assert hg1 is not hg2- assert hg1.empty() is True- assert hg2.empty() is False---class TestAutoHypergraphDeduplication:- """Test hyperedge deduplication."""-- def test_hyperedge_deduplication_ignores_participant_order(self, llm_client, embedder):- """Test that hyperedges with same participants in different order are deduplicated."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- # Create events with same event_type and participants but different order- e1 = Event(event_type="team_meeting", participants=["Alex", "Bailey", "Casey"])- e2 = Event(event_type="team_meeting", participants=["Casey", "Alex", "Bailey"])- e3 = Event(event_type="conference", participants=["Alex", "Bailey", "Casey"])-- # Manually add to edge memory to test deduplication- hg._edge_memory.add([e1, e2])-- # Should deduplicate e1 and e2 to 1 edge- assert hg._edge_memory.size == 1-- # Adding e3 should increase count (different event_type)- hg._edge_memory.add([e3])- assert hg._edge_memory.size == 2-- def test_multiple_hyperedges_with_different_participants(self, llm_client, embedder):- """Test that hyperedges with different participants are kept separate."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- e1 = Event(event_type="meeting", participants=["Alice", "Bob"])- e2 = Event(event_type="meeting", participants=["Alice", "Charlie"])-- hg._edge_memory.add([e1, e2])-- # Should have 2 different edges- assert hg._edge_memory.size == 2---class TestAutoHypergraphProperties:- """Test hypergraph data properties."""-- def test_nodes_property(self, llm_client, embedder):- """Test accessing nodes from hypergraph."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- hg.feed_text("""- Monthly Sync Meeting:- CEO John, CTO Lisa, and Product Manager Sarah discussed Q2 roadmap.- John, Lisa, and CFO David negotiated on budget allocation for infrastructure.- Lisa, Sarah, and QA Lead Tom reviewed test coverage requirements.- """)-- nodes = hg.data.nodes- assert isinstance(nodes, list)- assert all(isinstance(n, Person) for n in nodes)-- def test_edges_property(self, llm_client, embedder):- """Test accessing edges from hypergraph."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- hg.feed_text("""- Leadership Retreat Decisions:- Board Members Alice, Bob, and Charlie approved the new strategic plan.- Executives Alice, David, and Eve formed the new innovation committee.- Department heads Frank, Grace, and Henry committed to cross-team collaboration.- """)-- edges = hg.data.edges- assert isinstance(edges, list)- assert all(isinstance(e, Event) for e in edges)-- def test_edge_has_multiple_participants(self, llm_client, embedder):- """Test that hyperedges contain multiple participants."""- hg = AutoHypergraph(- node_schema=Person,- edge_schema=Event,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.event_type}_{sorted(x.participants)}",- nodes_in_edge_extractor=lambda x: tuple(x.participants),- llm_client=llm_client,- embedder=embedder,- )-- event = Event(event_type="workshop", participants=["Person1", "Person2", "Person3", "Person4"])- hg._edge_memory.add([event])-- assert len(hg.data.edges) > 0- assert len(hg.data.edges[0].participants) >= 2diff --git a/tests/types/test_list.py b/tests/types/test_list.pydeleted file mode 100644index 752fa32..0000000--- a/tests/types/test_list.py+++ /dev/null@@ -1,220 +0,0 @@-"""Tests for AutoList - list extraction pattern."""--import pytest-from pydantic import BaseModel, Field-from hyperextract.types.list import AutoList---class Task(BaseModel):- """Test schema for list items."""-- title: str = Field(default="", description="Task title")- priority: int = Field(default=1)- done: bool = Field(default=False)---class Person(BaseModel):- """Test schema for people."""-- name: str = Field(default="", description="Person's name")- age: int = Field(default=0)---class TestAutoListBasics:- """Test basic AutoList functionality."""-- def test_initialization(self, llm_client, embedder):- """Test AutoList initialization."""- task_list = AutoList(- item_schema=Task, llm_client=llm_client, embedder=embedder- )-- assert task_list.item_schema == Task- assert task_list.empty()- assert len(task_list) == 0-- def test_properties(self, llm_client, embedder):- """Test AutoList properties."""- task_list = AutoList(Task, llm_client, embedder)-- # Check items property- assert task_list.items == []- assert isinstance(task_list.items, list)---class TestAutoListFeed:- """Test feeding text to AutoList."""-- def test_feed_text_populates_list(self, llm_client, embedder):- """Test feeding text extracts items into list."""- task_list = AutoList(Task, llm_client, embedder)-- task_list.feed_text("Buy milk and walk the dog")-- assert not task_list.empty()- assert len(task_list) > 0- assert isinstance(task_list[0], Task)-- def test_feed_text_appends(self, llm_client, embedder):- """Test that feed_text appends, not replaces."""- task_list = AutoList(Task, llm_client, embedder)-- # Manually set initial items- task_list._data.items = [Task(title="Item 1", priority=1)]- initial_len = len(task_list)-- # Feed more text (in mock mode, this adds mock items)- task_list.feed_text("""- Q1 2024 Sprint Tasks:- 1. Implement user authentication (High priority)- 2. Optimize database queries (Medium priority)- 3. Write API documentation (Low priority)- 4. Security audit review (Critical priority)- 5. Deploy staging environment (High priority)- """)-- # Should have more items- assert len(task_list) >= initial_len---class TestAutoListIndexing:- """Test indexing and slicing operations."""-- def test_integer_indexing(self, llm_client, embedder):- """Test integer indexing returns single item."""- task_list = AutoList(Task, llm_client, embedder)-- t1 = Task(title="First", priority=1)- t2 = Task(title="Second", priority=2)- t3 = Task(title="Third", priority=3)- task_list._data.items = [t1, t2, t3]-- assert task_list[0] == t1- assert task_list[1] == t2- assert task_list[-1] == t3-- def test_out_of_bounds_indexing(self, llm_client, embedder):- """Test IndexError for out of bounds access."""- task_list = AutoList(Task, llm_client, embedder)- task_list._data.items = [Task(title="Only", priority=1)]-- with pytest.raises(IndexError):- _ = task_list[10]-- def test_slice_returns_new_list(self, llm_client, embedder):- """Test slicing returns new AutoList instance."""- task_list = AutoList(Task, llm_client, embedder)-- t1 = Task(title="First", priority=1)- t2 = Task(title="Second", priority=2)- t3 = Task(title="Third", priority=3)- task_list._data.items = [t1, t2, t3]-- # Test slice- sliced = task_list[0:2]-- assert isinstance(sliced, AutoList)- assert sliced.item_schema == Task- assert len(sliced) == 2- assert sliced[0] == t1- assert sliced[1] == t2-- def test_slice_with_step(self, llm_client, embedder):- """Test slicing with step."""- task_list = AutoList(Task, llm_client, embedder)-- items = [Task(title=f"Item {i}", priority=i) for i in range(5)]- task_list._data.items = items-- sliced = task_list[::2]- assert len(sliced) == 3---class TestAutoListSetItem:- """Test item assignment."""-- def test_setitem_updates_item(self, llm_client, embedder):- """Test __setitem__ replaces item at index."""- task_list = AutoList(Task, llm_client, embedder)-- t1 = Task(title="First")- t2 = Task(title="Second")- task_list._data.items = [t1, t2]-- new_item = Task(title="Updated")- task_list[0] = new_item-- assert task_list[0].title == "Updated"- assert task_list[1] == t2-- def test_setitem_invalid_schema(self, llm_client, embedder):- """Test that setting wrong schema raises error."""- task_list = AutoList(Task, llm_client, embedder)- task_list._data.items = [Task(title="Item")]-- with pytest.raises(TypeError):- task_list[0] = Person(name="Not a Task")---class TestAutoListAddition:- """Test list concatenation."""-- def test_list_plus_list(self, llm_client, embedder):- """Test AutoList + AutoList merges both lists."""- list1 = AutoList(Task, llm_client, embedder)- list1._data.items = [Task(title="A", priority=1)]-- list2 = AutoList(Task, llm_client, embedder)- list2._data.items = [Task(title="B", priority=2)]-- result = list1 + list2-- assert isinstance(result, AutoList)- assert len(result) == 2- assert result[0].title == "A"- assert result[1].title == "B"-- def test_list_plus_model(self, llm_client, embedder):- """Test AutoList + AutoModel appends model."""- from hyperextract.types.model import AutoModel-- list_obj = AutoList(Task, llm_client, embedder)- list_obj._data.items = [Task(title="Existing", priority=1)]-- model = AutoModel(Task, llm_client, embedder)- model._data = Task(title="New Item", priority=2)-- result = list_obj + model-- assert isinstance(result, AutoList)- assert len(result) == 2- assert result[1].title == "New Item"---class TestAutoListDeletion:- """Test item deletion."""-- def test_delitem(self, llm_client, embedder):- """Test __delitem__ removes item at index."""- task_list = AutoList(Task, llm_client, embedder)-- items = [Task(title=f"Item {i}") for i in range(3)]- task_list._data.items = items-- del task_list[1]-- assert len(task_list) == 2- assert task_list[0].title == "Item 0"- assert task_list[1].title == "Item 2"---class TestAutoListLen:- """Test length operations."""-- def test_len(self, llm_client, embedder):- """Test __len__ returns list length."""- task_list = AutoList(Task, llm_client, embedder)- assert len(task_list) == 0-- task_list._data.items = [Task(title=str(i)) for i in range(5)]- assert len(task_list) == 5diff --git a/tests/types/test_list_iteration.py b/tests/types/test_list_iteration.pynew file mode 100644index 0000000..df9e7dc--- /dev/null+++ b/tests/types/test_list_iteration.py@@ -0,0 +1,268 @@+"""Unit tests for AutoList iteration and membership."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoList+from tests.fixtures import PersonSchema+++class PersonItemSchema(BaseModel):+ """Schema for person list items."""++ name: str+ age: Optional[int] = None+++class TestAutoListIteration:+ """Test cases for AutoList iteration and membership."""++ def test_iteration(self, llm_client, embedder):+ """Test that AutoList is iterable."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ names = []+ for item in auto_list:+ names.append(item.name)++ assert names == ["John", "Jane"]++ def test_iteration_with_list_comprehension(self, llm_client, embedder):+ """Test iteration with list comprehension."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ names = [item.name for item in auto_list]++ assert names == ["John", "Jane"]++ def test_list_function(self, llm_client, embedder):+ """Test converting to Python list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ items_list = list(auto_list)++ assert len(items_list) == 2+ assert isinstance(items_list, list)++ def test_contains_true(self, llm_client, embedder):+ """Test that contains returns True for existing item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item = PersonItemSchema(name="John", age=30)+ auto_list.append(item)++ assert item in auto_list++ def test_contains_false(self, llm_client, embedder):+ """Test that contains returns False for non-existing item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ assert PersonItemSchema(name="NotFound") not in auto_list++ def test_contains_with_same_values(self, llm_client, embedder):+ """Test contains with items that have same values."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item1 = PersonItemSchema(name="John", age=30)+ auto_list.append(item1)++ item2 = PersonItemSchema(name="John", age=30)+ assert item2 in auto_list++ def test_not_in_operator(self, llm_client, embedder):+ """Test not in operator."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ assert PersonItemSchema(name="NotFound") not in auto_list+++class TestAutoListOperators:+ """Test cases for AutoList operators."""++ def test_add_two_lists(self, llm_client, embedder):+ """Test adding two AutoList instances."""+ list1 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ list2 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonItemSchema(name="John"))+ list2.append(PersonItemSchema(name="Jane"))++ result = list1 + list2++ assert len(result) == 2+ assert result.items[0].name == "John"+ assert result.items[1].name == "Jane"++ def test_add_preserves_original(self, llm_client, embedder):+ """Test that add preserves original lists."""+ list1 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ list2 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonItemSchema(name="John"))+ list2.append(PersonItemSchema(name="Jane"))++ result = list1 + list2++ assert len(list1) == 1+ assert len(list2) == 1+ assert len(result) == 2++ def test_add_chaining(self, llm_client, embedder):+ """Test chaining multiple add operations."""+ list1 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ list2 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ list3 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonItemSchema(name="John"))+ list2.append(PersonItemSchema(name="Jane"))+ list3.append(PersonItemSchema(name="Bob"))++ result = list1 + list2 + list3++ assert len(result) == 3++ def test_add_with_different_schema_raises(self, llm_client, embedder):+ """Test that adding lists with different schemas raises TypeError."""+ from hyperextract.types import AutoModel++ list1 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list2 = AutoList(+ item_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonItemSchema(name="John"))+ list2.append(PersonSchema(name="Jane", age=30))++ with pytest.raises(TypeError):+ list1 + list2++ def test_add_with_automodel(self, llm_client, embedder):+ """Test adding AutoList with AutoModel."""+ from hyperextract.types import AutoModel++ list1 = AutoList(+ item_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonSchema(name="John", age=30))++ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ model.feed_text("Jane is 25 years old.")++ result = list1 + model++ assert len(result) == 2++ def test_setitem(self, llm_client, embedder):+ """Test setting item by index."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ auto_list[0] = PersonItemSchema(name="Bob", age=35)++ assert auto_list.items[0].name == "Bob"++ def test_delitem(self, llm_client, embedder):+ """Test deleting item by index."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ del auto_list[0]++ assert len(auto_list) == 1+ assert auto_list.items[0].name == "Jane"diff --git a/tests/types/test_list_operations.py b/tests/types/test_list_operations.pynew file mode 100644index 0000000..d47c5b2--- /dev/null+++ b/tests/types/test_list_operations.py@@ -0,0 +1,299 @@+"""Unit tests for AutoList operations."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoList+from tests.fixtures import PersonSchema+++class PersonItemSchema(BaseModel):+ """Schema for person list items."""++ name: str+ age: Optional[int] = None+++class TestAutoListOperations:+ """Test cases for AutoList basic operations."""++ def test_init_and_empty_state(self, llm_client, embedder):+ """Test that AutoList initializes correctly."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert auto_list.empty() is True+ assert len(auto_list) == 0+ assert len(auto_list.items) == 0++ def test_item_schema_property(self, llm_client, embedder):+ """Test that item_schema property returns correct schema."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert auto_list.item_schema == PersonItemSchema++ def test_append_single_item(self, llm_client, embedder):+ """Test appending a single item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item = PersonItemSchema(name="John", age=30)+ auto_list.append(item)++ assert len(auto_list) == 1+ assert auto_list.items[0].name == "John"++ def test_append_multiple_items(self, llm_client, embedder):+ """Test appending multiple items one by one."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ assert len(auto_list) == 2++ def test_extend_with_list(self, llm_client, embedder):+ """Test extending with a Python list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ items = [+ PersonItemSchema(name="John", age=30),+ PersonItemSchema(name="Jane", age=25),+ ]+ auto_list.extend(items)++ assert len(auto_list) == 2++ def test_extend_with_autolist(self, llm_client, embedder):+ """Test extending with another AutoList."""+ list1 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )+ list2 = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ list1.append(PersonItemSchema(name="John"))+ list2.append(PersonItemSchema(name="Jane"))++ list1.extend(list2)++ assert len(list1) == 2++ def test_insert_at_beginning(self, llm_client, embedder):+ """Test inserting at the beginning of the list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.insert(0, PersonItemSchema(name="Jane"))++ assert auto_list.items[0].name == "Jane"+ assert auto_list.items[1].name == "John"++ def test_insert_at_end(self, llm_client, embedder):+ """Test inserting at the end of the list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.insert(1, PersonItemSchema(name="Jane"))++ assert auto_list.items[1].name == "Jane"++ def test_remove_item(self, llm_client, embedder):+ """Test removing an item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item = PersonItemSchema(name="John", age=30)+ auto_list.append(item)+ auto_list.append(PersonItemSchema(name="Jane"))++ auto_list.remove(item)++ assert len(auto_list) == 1+ assert auto_list.items[0].name == "Jane"++ def test_remove_raises_when_not_found(self, llm_client, embedder):+ """Test that remove raises ValueError when item not found."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))++ with pytest.raises(ValueError):+ auto_list.remove(PersonItemSchema(name="NotFound"))++ def test_pop_default(self, llm_client, embedder):+ """Test popping from end by default."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))++ popped = auto_list.pop()++ assert popped.name == "Jane"+ assert len(auto_list) == 1++ def test_pop_first(self, llm_client, embedder):+ """Test popping from beginning."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))++ popped = auto_list.pop(0)++ assert popped.name == "John"+ assert len(auto_list) == 1++ def test_pop_from_empty_raises(self, llm_client, embedder):+ """Test that popping from empty list raises IndexError."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ with pytest.raises(IndexError):+ auto_list.pop()++ def test_clear(self, llm_client, embedder):+ """Test clearing the list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))++ assert len(auto_list) == 2++ auto_list.clear()++ assert len(auto_list) == 0+ assert auto_list.empty() is True++ def test_index_method(self, llm_client, embedder):+ """Test finding index of an item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))++ idx = auto_list.index(PersonItemSchema(name="Jane"))++ assert idx == 1++ def test_index_raises_when_not_found(self, llm_client, embedder):+ """Test that index raises ValueError when item not found."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))++ with pytest.raises(ValueError):+ auto_list.index(PersonItemSchema(name="NotFound"))++ def test_count_method(self, llm_client, embedder):+ """Test counting occurrences of an item."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))+ auto_list.append(PersonItemSchema(name="John"))++ count = auto_list.count(PersonItemSchema(name="John"))++ assert count == 2++ def test_copy(self, llm_client, embedder):+ """Test copying the list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))++ copied = auto_list.copy()++ assert len(copied) == 1+ assert copied.items[0].name == "John"+ assert copied.items[0] is not auto_list.items[0]++ def test_reverse(self, llm_client, embedder):+ """Test reversing the list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))+ auto_list.append(PersonItemSchema(name="Jane"))+ auto_list.append(PersonItemSchema(name="Bob"))++ auto_list.reverse()++ assert auto_list.items[0].name == "Bob"+ assert auto_list.items[2].name == "John"diff --git a/tests/types/test_list_slicing.py b/tests/types/test_list_slicing.pynew file mode 100644index 0000000..cb11596--- /dev/null+++ b/tests/types/test_list_slicing.py@@ -0,0 +1,278 @@+"""Unit tests for AutoList slicing and indexing."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoList+++class PersonItemSchema(BaseModel):+ """Schema for person list items."""++ name: str+ age: Optional[int] = None+++class TestAutoListIndexing:+ """Test cases for AutoList indexing."""++ def test_getitem_positive_index(self, llm_client, embedder):+ """Test getting item by positive index."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))+ auto_list.append(PersonItemSchema(name="Bob", age=35))++ item = auto_list[0]++ assert item.name == "John"++ def test_getitem_negative_index(self, llm_client, embedder):+ """Test getting item by negative index."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))+ auto_list.append(PersonItemSchema(name="Bob", age=35))++ item = auto_list[-1]++ assert item.name == "Bob"++ def test_getitem_out_of_range_raises(self, llm_client, embedder):+ """Test that getting item out of range raises IndexError."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John"))++ with pytest.raises(IndexError):+ _ = auto_list[10]++ def test_getitem_empty_list_raises(self, llm_client, embedder):+ """Test that getting item from empty list raises IndexError."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ with pytest.raises(IndexError):+ _ = auto_list[0]+++class TestAutoListSlicing:+ """Test cases for AutoList slicing."""++ def test_slice_basic(self, llm_client, embedder):+ """Test basic slicing."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))+ auto_list.append(PersonItemSchema(name="Bob", age=35))++ sliced = auto_list[0:2]++ assert isinstance(sliced, AutoList)+ assert len(sliced) == 2+ assert sliced.items[0].name == "John"+ assert sliced.items[1].name == "Jane"++ def test_slice_returns_new_instance(self, llm_client, embedder):+ """Test that slicing returns a new instance."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ sliced = auto_list[0:1]++ assert sliced is not auto_list++ def test_slice_does_not_modify_original(self, llm_client, embedder):+ """Test that slicing does not modify the original list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ sliced = auto_list[0:1]++ assert len(auto_list) == 2++ def test_slice_first_n(self, llm_client, embedder):+ """Test slicing first n items."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ for i in range(5):+ auto_list.append(PersonItemSchema(name=f"Person{i}", age=20 + i))++ sliced = auto_list[:3]++ assert len(sliced) == 3+ assert sliced.items[0].name == "Person0"+ assert sliced.items[2].name == "Person2"++ def test_slice_last_n(self, llm_client, embedder):+ """Test slicing last n items."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ for i in range(5):+ auto_list.append(PersonItemSchema(name=f"Person{i}", age=20 + i))++ sliced = auto_list[-2:]++ assert len(sliced) == 2+ assert sliced.items[0].name == "Person3"+ assert sliced.items[1].name == "Person4"++ def test_slice_with_step(self, llm_client, embedder):+ """Test slicing with step."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ for i in range(6):+ auto_list.append(PersonItemSchema(name=f"Person{i}", age=20 + i))++ sliced = auto_list[::2]++ assert len(sliced) == 3+ assert sliced.items[0].name == "Person0"+ assert sliced.items[1].name == "Person2"+ assert sliced.items[2].name == "Person4"++ def test_slice_with_negative_step(self, llm_client, embedder):+ """Test slicing with negative step (reverses list)."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))+ auto_list.append(PersonItemSchema(name="Bob", age=35))++ sliced = auto_list[::-1]++ assert sliced.items[0].name == "Bob"+ assert sliced.items[2].name == "John"++ def test_slice_empty_result(self, llm_client, embedder):+ """Test slicing that results in empty list."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ sliced = auto_list[5:10]++ assert isinstance(sliced, AutoList)+ assert len(sliced) == 0++ def test_slice_preserves_schema(self, llm_client, embedder):+ """Test that sliced list preserves item schema."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ sliced = auto_list[:]++ assert sliced.item_schema == auto_list.item_schema++ def test_sliced_list_is_independent(self, llm_client, embedder):+ """Test that sliced list is independent of original."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ sliced = auto_list[:]++ sliced.append(PersonItemSchema(name="Jane", age=25))++ assert len(auto_list) == 1+ assert len(sliced) == 2+++class TestAutoListRepr:+ """Test cases for AutoList string representation."""++ def test_repr(self, llm_client, embedder):+ """Test __repr__ method."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))+ auto_list.append(PersonItemSchema(name="Jane", age=25))++ repr_str = repr(auto_list)++ assert "AutoList" in repr_str+ assert "PersonItemSchema" in repr_str+ assert "2" in repr_str++ def test_str(self, llm_client, embedder):+ """Test __str__ method."""+ auto_list = AutoList(+ item_schema=PersonItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ auto_list.append(PersonItemSchema(name="John", age=30))++ str_repr = str(auto_list)++ assert "AutoList" in str_repr+ assert "1" in str_reprdiff --git a/tests/types/test_model.py b/tests/types/test_model.pydeleted file mode 100644index 9246b16..0000000--- a/tests/types/test_model.py+++ /dev/null@@ -1,223 +0,0 @@-"""Tests for AutoModel - single object extraction pattern."""--import pytest-from pydantic import BaseModel, Field-from datetime import datetime-from hyperextract.types.model import AutoModel-from ontomem.merger import MergeStrategy---class Person(BaseModel):- """Test schema for AutoModel tests."""-- name: str | None = Field(default=None, description="Person's name")- age: int | None = Field(default=None, description="Person's age")- profession: str | None = Field(default=None, description="Person's profession")---class Article(BaseModel):- """Test schema for article summary."""-- title: str = Field(default="", description="Article title")- author: str = Field(default="", description="Article author")- summary: str = Field(default="", description="Article summary")---class TestAutoModelBasics:- """Test basic AutoModel functionality."""-- def test_initialization(self, llm_client, embedder):- """Test AutoModel initialization."""- model = AutoModel(- data_schema=Person,- llm_client=llm_client,- embedder=embedder,- strategy_or_merger=MergeStrategy.MERGE_FIELD,- )-- assert model.data_schema == Person- # Note: empty() may return False if default values are present- assert isinstance(model.data, Person)-- def test_properties(self, llm_client, embedder):- """Test AutoModel properties."""- model = AutoModel(Person, llm_client, embedder)-- # Check metadata- assert "created_at" in model.metadata- assert "updated_at" in model.metadata- assert isinstance(model.metadata["created_at"], datetime)-- def test_empty_check(self, llm_client, embedder):- """Test empty() method for detecting uninitialized data."""- model = AutoModel(Person, llm_client, embedder)- # Initially, model may or may not be empty depending on default values- initial_state = model.empty()-- # Explicitly set non-default values- model._data.name = "John Doe"- # After setting explicit data, should not be empty- assert model.empty() is False---class TestAutoModelExtraction:- """Test extraction and data management."""-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test that extract() returns a new instance."""- model1 = AutoModel(Person, llm_client, embedder)- model2 = model1.parse("""Dr. James Wilson, 47 years old, served as Chief Technology Officer- at Innovate Systems for 12 years before joining CloudFirst as VP of Engineering.""")-- # Should be different instances- assert model1 is not model2- # Original should still be empty- assert model1.empty() is True- # New should have data- assert not model2.empty()-- def test_feed_text_updates_in_place(self, llm_client, embedder):- """Test that feed_text() updates current instance."""- model = AutoModel(Person, llm_client, embedder)- original_id = id(model)-- result = model.feed_text("""- Dr. Sarah Chen is a 42-year-old data scientist at Tech Labs.- She specializes in machine learning and supervises 8 engineers.- Sarah earned her PhD from Stanford in 2008.- """)-- # Should return self- assert id(result) == original_id- # Should not be empty- assert not model.empty()-- def test_feed_text_chaining(self, llm_client, embedder):- """Test that feed_text() supports method chaining."""- model = AutoModel(Person, llm_client, embedder)-- result = model.feed_text(- "Alexandra Martinez is a 35-year-old software architect"- ).feed_text(- "She works at TechCorp and has 12 years of experience in distributed systems"- )-- assert result is model- assert not model.empty()---class TestAutoModelClear:- """Test clearing operations."""-- def test_clear_resets_data(self, llm_client, embedder):- """Test clear() resets all data."""- model = AutoModel(Person, llm_client, embedder)- model.feed_text("""- Prof. Michael Johnson is a 56-year-old cloud architect.- He has 20 years of experience in distributed systems.- Michael leads the infrastructure team.- """)- assert not model.empty()-- model.clear()- assert model.empty() is True-- def test_clear_updates_metadata(self, llm_client, embedder):- """Test that clear() updates metadata."""- model = AutoModel(Person, llm_client, embedder)- old_time = model.metadata["updated_at"]-- model.feed_text("""- Emma Rodriguez is a 38-year-old product manager at StartupCorp.- She specializes in B2B SaaS products and leads cross-functional teams.- Emma worked at Microsoft for 10 years before joining StartupCorp.- """)- model.clear()-- # Updated time should be newer- assert model.metadata["updated_at"] >= old_time---class TestAutoModelOperators:- """Test operator overloading."""-- def test_model_plus_model_creates_list(self, llm_client, embedder):- """Test AutoModel + AutoModel creates AutoList."""- from hyperextract.types.list import AutoList-- model1 = AutoModel(Person, llm_client, embedder)- model1._data = Person(name="Alice", age=30, profession="Engineer")-- model2 = AutoModel(Person, llm_client, embedder)- model2._data = Person(name="Bob", age=25, profession="Designer")-- result = model1 + model2-- assert isinstance(result, AutoList)- assert result.item_schema == Person- assert len(result) == 2-- def test_model_plus_model_same_schema_required(self, llm_client, embedder):- """Test that + operator requires same schema."""- model1 = AutoModel(Person, llm_client, embedder)-- article_model = AutoModel(Article, llm_client, embedder)-- with pytest.raises(TypeError, match="different schemas"):- model1 + article_model-- def test_model_plus_list_creates_list(self, llm_client, embedder):- """Test AutoModel + AutoList prepends model to list."""- from hyperextract.types.list import AutoList-- model = AutoModel(Person, llm_client, embedder)- model._data = Person(name="Charlie", age=35)-- auto_list = AutoList(Person, llm_client, embedder)- person1 = Person(name="Alice", age=30)- person2 = Person(name="Bob", age=25)- auto_list._data.items = [person1, person2]-- result = model + auto_list-- assert isinstance(result, AutoList)- assert len(result) == 3- # Model should be prepended- assert result[0].name == "Charlie"- assert result[1].name == "Alice"- assert result[2].name == "Bob"---class TestAutoModelMerging:- """Test merge batch data functionality."""-- def test_merge_batch_multiple_items(self, llm_client, embedder):- """Test merging multiple extracted items."""- # Use MERGE_FIELD strategy to test deterministic field-level merging- # without depending on LLM (works with or without API key)- model = AutoModel(Person, llm_client, embedder, strategy_or_merger=MergeStrategy.MERGE_FIELD)-- # Create mock items to merge- item1 = Person(name="John", age=30, profession="Engineer")- item2 = Person(name="Jane", age=28, profession="Doctor")-- # Merge strategy: MERGE_FIELD merges non-null fields- result = model.merge_batch_data([item1, item2])-- # Result should be a valid Person with one of the merged data- assert isinstance(result, Person)- # Should have a name (from one of the items)- assert result.name in ["John", "Jane"]- # Should have age and profession (from merged data)- assert result.age is not None- assert result.profession is not None-- def test_merge_batch_empty_list(self, llm_client, embedder):- """Test merging empty list."""- model = AutoModel(Person, llm_client, embedder)-- result = model.merge_batch_data([])-- # Should return empty instance- assert isinstance(result, Person)- assert result.model_dump(exclude_unset=True) == {}diff --git a/tests/types/test_model_basic.py b/tests/types/test_model_basic.pynew file mode 100644index 0000000..29866a5--- /dev/null+++ b/tests/types/test_model_basic.py@@ -0,0 +1,129 @@+"""Unit tests for AutoModel basic functionality."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoModel+from tests.fixtures import PersonSchema, SHORT_TEXT+++class TestAutoModelBasic:+ """Test cases for AutoModel basic functionality."""++ def test_init_and_empty_state(self, llm_client, embedder):+ """Test that AutoModel initializes correctly and reports empty state."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert model.empty() is True+ assert model.data is None++ def test_data_schema_property(self, llm_client, embedder):+ """Test that data_schema property returns correct schema."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert model.data_schema == PersonSchema++ def test_parse_returns_new_instance(self, llm_client, embedder):+ """Test that parse() returns a new instance with data."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ parsed = model.parse(SHORT_TEXT)++ assert parsed is not model+ assert parsed.empty() is False+ assert parsed.data is not None++ def test_feed_text_updates_state(self, llm_client, embedder):+ """Test that feed_text() updates the internal state."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert model.empty() is True++ model.feed_text(SHORT_TEXT)++ assert model.empty() is False+ assert model.data is not None++ def test_feed_text_returns_self(self, llm_client, embedder):+ """Test that feed_text() returns self for method chaining."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ result = model.feed_text(SHORT_TEXT)++ assert result is model++ def test_clear_resets_state(self, llm_client, embedder):+ """Test that clear() resets the model to empty state."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text(SHORT_TEXT)+ assert model.empty() is False++ model.clear()+ assert model.empty() is True++ def test_metadata_initialized(self, llm_client, embedder):+ """Test that metadata is initialized correctly."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert "created_at" in model.metadata+ assert "updated_at" in model.metadata+ assert model.metadata["created_at"] <= model.metadata["updated_at"]++ def test_metadata_updated_on_feed(self, llm_client, embedder):+ """Test that metadata is updated when feeding text."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ original_updated_at = model.metadata["updated_at"]++ model.feed_text(SHORT_TEXT)++ assert model.metadata["updated_at"] > original_updated_at++ def test_create_empty_instance(self, llm_client, embedder):+ """Test that _create_empty_instance creates a properly configured instance."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ chunk_size=1024,+ )++ empty_instance = model._create_empty_instance()++ assert empty_instance.data_schema == model.data_schema+ assert empty_instance.chunk_size == model.chunk_size+ assert empty_instance.empty() is Truediff --git a/tests/types/test_model_merge.py b/tests/types/test_model_merge.pynew file mode 100644index 0000000..1b91f89--- /dev/null+++ b/tests/types/test_model_merge.py@@ -0,0 +1,129 @@+"""Unit tests for AutoModel merge functionality."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoModel+from tests.fixtures import BiographySchema+++class SimpleSchema(BaseModel):+ """Simple schema for testing merge functionality."""++ name: str+ age: Optional[int] = None+ occupation: Optional[str] = None+++class TestAutoModelMerge:+ """Test cases for AutoModel merge functionality."""++ def test_merge_batch_data_single_item(self, llm_client, embedder):+ """Test merge_batch_data with a single item."""+ model = AutoModel(+ data_schema=SimpleSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item = SimpleSchema(name="John", age=30)+ result = model.merge_batch_data([item])++ assert result.name == "John"+ assert result.age == 30++ def test_merge_batch_data_empty_list(self, llm_client, embedder):+ """Test merge_batch_data with empty list."""+ model = AutoModel(+ data_schema=SimpleSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ result = model.merge_batch_data([])++ assert result is None++ def test_merge_batch_data_multiple_items(self, llm_client, embedder):+ """Test merge_batch_data with multiple items."""+ model = AutoModel(+ data_schema=SimpleSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ item1 = SimpleSchema(name="John", age=30)+ item2 = SimpleSchema(name="John", age=31, occupation="Engineer")+ item3 = SimpleSchema(name="John", occupation="Developer")++ result = model.merge_batch_data([item1, item2, item3])++ assert result.name == "John"++ def test_feed_text_multiple_times(self, llm_client, embedder):+ """Test that feeding text multiple times updates the data."""+ model = AutoModel(+ data_schema=SimpleSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John is 30 years old.")+ first_data = model.data++ model.feed_text("John works as an engineer.")+ second_data = model.data++ assert second_data is not None++ def test_parse_creates_separate_instance(self, llm_client, embedder):+ """Test that parse() creates instances that don't share state."""+ model = AutoModel(+ data_schema=SimpleSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ parsed1 = model.parse("John is 30 years old.")+ parsed2 = model.parse("Jane is 25 years old.")++ assert parsed1.data is not parsed2.data+++class TestAutoModelWithComplexSchema:+ """Test AutoModel with more complex schemas."""++ def test_with_biography_schema(self, llm_client, embedder):+ """Test AutoModel with BiographySchema."""+ model = AutoModel(+ data_schema=BiographySchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ result = model.parse(+ "Albert Einstein was born in 1879 in Ulm, Germany. "+ "He was a theoretical physicist who developed the theory of relativity."+ )++ assert result.empty() is False+ assert result.data.name == "Albert Einstein"++ def test_empty_optional_fields(self, llm_client, embedder):+ """Test that empty optional fields are handled correctly."""++ class MinimalSchema(BaseModel):+ name: str+ description: Optional[str] = None++ model = AutoModel(+ data_schema=MinimalSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ parsed = model.parse("Someone named John.")++ assert parsed.empty() is False+ assert parsed.data.name is not Nonediff --git a/tests/types/test_model_serialization.py b/tests/types/test_model_serialization.pynew file mode 100644index 0000000..a159bff--- /dev/null+++ b/tests/types/test_model_serialization.py@@ -0,0 +1,178 @@+"""Unit tests for AutoModel serialization functionality."""++import json+import tempfile+import shutil+from pathlib import Path++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoModel+from tests.fixtures import PersonSchema+++class TestAutoModelSerialization:+ """Test cases for AutoModel serialization and deserialization."""++ def setup_method(self):+ """Create a temporary directory for each test."""+ self.temp_dir = tempfile.mkdtemp()++ def teardown_method(self):+ """Clean up temporary directory after each test."""+ if Path(self.temp_dir).exists():+ shutil.rmtree(self.temp_dir)++ def test_dump_data_creates_file(self, llm_client, embedder):+ """Test that dump_data() creates a JSON file."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer at TechCorp.")++ file_path = Path(self.temp_dir) / "data.json"+ model.dump_data(file_path)++ assert file_path.exists()++ def test_dump_data_contains_correct_data(self, llm_client, embedder):+ """Test that dumped data contains correct information."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer at TechCorp.")++ file_path = Path(self.temp_dir) / "data.json"+ model.dump_data(file_path)++ with open(file_path, "r", encoding="utf-8") as f:+ data = json.load(f)++ assert "name" in data+ assert data["name"] == "John Smith"++ def test_load_data_restores_state(self, llm_client, embedder):+ """Test that load_data() restores the model state."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer at TechCorp.")++ data_file = Path(self.temp_dir) / "data.json"+ model.dump_data(data_file)++ new_model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ assert new_model.empty() is True++ new_model.load_data(data_file)++ assert new_model.empty() is False++ def test_load_data_raises_on_missing_file(self, llm_client, embedder):+ """Test that load_data() raises FileNotFoundError for missing files."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ with pytest.raises(FileNotFoundError):+ model.load_data(Path(self.temp_dir) / "nonexistent.json")++ def test_dump_metadata(self, llm_client, embedder):+ """Test that dump_metadata() works correctly."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ file_path = Path(self.temp_dir) / "metadata.json"+ model.dump_metadata(file_path)++ assert file_path.exists()++ with open(file_path, "r", encoding="utf-8") as f:+ metadata = json.load(f)++ assert "created_at" in metadata+ assert "updated_at" in metadata++ def test_load_metadata(self, llm_client, embedder):+ """Test that load_metadata() restores metadata."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer at TechCorp.")++ metadata_file = Path(self.temp_dir) / "metadata.json"+ model.dump_metadata(metadata_file)++ new_model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ new_model.load_metadata(metadata_file)++ assert "created_at" in new_model.metadata+ assert "updated_at" in new_model.metadata++ def test_dump_and_load_full(self, llm_client, embedder):+ """Test full dump/load cycle."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer at TechCorp.")++ model.dump(self.temp_dir)++ new_model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ new_model.load(self.temp_dir)++ assert new_model.empty() is False+ assert new_model.data.name == "John Smith"++ def test_dump_creates_directory(self, llm_client, embedder):+ """Test that dump() creates the directory if it doesn't exist."""+ model = AutoModel(+ data_schema=PersonSchema,+ llm_client=llm_client,+ embedder=embedder,+ )++ model.feed_text("John Smith is a software engineer.")++ new_dir = Path(self.temp_dir) / "new_subdir" / "nested"+ model.dump(new_dir)++ assert new_dir.exists()+ assert (new_dir / "data.json").exists()diff --git a/tests/types/test_set.py b/tests/types/test_set.pydeleted file mode 100644index efdff72..0000000--- a/tests/types/test_set.py+++ /dev/null@@ -1,316 +0,0 @@-"""Tests for AutoSet - set extraction pattern with deduplication."""--import pytest-from pydantic import BaseModel, Field-from ontomem.merger import MergeStrategy-from hyperextract.types.set import AutoSet---class BookEntry(BaseModel):- """Test schema for book entries."""-- title: str = Field(description="Book title (unique key)")- author: str | None = Field(None, description="Book author")- year: int | None = Field(None, description="Publication year")---class TestAutoSetBasics:- """Test basic AutoSet functionality."""-- def test_initialization(self, llm_client, embedder):- """Test AutoSet initialization."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- strategy_or_merger=MergeStrategy.MERGE_FIELD,- )-- assert books.item_schema == BookEntry- assert books.empty()- assert len(books) == 0- assert books.keys == []-- def test_key_extractor_validation(self, llm_client, embedder):- """Test that key_extractor is properly stored."""- key_func = lambda x: x.title-- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=key_func,- )-- assert books.key_extractor == key_func---class TestAutoSetDeduplication:- """Test deduplication core functionality."""-- def test_add_deduplicates_same_key(self, llm_client, embedder):- """Test that adding items with same key deduplicates."""- # Use MERGE_FIELD strategy to test deterministic deduplication- # without depending on LLM (works with or without API key)- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- strategy_or_merger=MergeStrategy.MERGE_FIELD,- )-- b1 = BookEntry(title="Python Basics", author="John", year=2020)- books.add(b1)- assert len(books) == 1-- # Add same book with different author info- b2 = BookEntry(title="Python Basics", author="John Smith", year=2021)- books.add(b2)-- # Should still be 1 item (deduplicated with MERGE_FIELD strategy)- # MERGE_FIELD merges non-null fields, so both books merge into one- assert len(books) == 1-- def test_add_keeps_distinct_keys(self, llm_client, embedder):- """Test that distinct keys are kept."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.add(BookEntry(title="Python Basics"))- books.add(BookEntry(title="Web Development"))- books.add(BookEntry(title="Data Science"))-- assert len(books) == 3- assert "Python Basics" in books- assert "Web Development" in books- assert "Data Science" in books-- def test_keys_property(self, llm_client, embedder):- """Test keys property returns all unique keys."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.add(BookEntry(title="Advanced Python"))- books.add(BookEntry(title="Web Frameworks"))- books.add(BookEntry(title="ML Fundamentals"))-- keys = books.keys- assert len(keys) == 3- assert "Advanced Python" in keys- assert "Web Frameworks" in keys- assert "ML Fundamentals" in keys---class TestAutoSetOperations:- """Test set-like operations."""-- def test_contains_operator(self, llm_client, embedder):- """Test 'in' operator (contains)."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.add(BookEntry(title="Python Pro"))-- assert "Python Pro" in books- assert "Java Expert" not in books-- def test_contains_method(self, llm_client, embedder):- """Test contains method."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.add(BookEntry(title="Go Guide"))-- assert books.contains("Go Guide") is True- assert books.contains("Rust Book") is False-- def test_get_method(self, llm_client, embedder):- """Test get method retrieves items by key."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- book = BookEntry(title="TypeScript Handbook", author="Microsoft", year=2022)- books.add(book)-- retrieved = books.get("TypeScript Handbook")- assert retrieved is not None- assert retrieved.title == "TypeScript Handbook"-- def test_get_with_default(self, llm_client, embedder):- """Test get with default value."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- result = books.get("Nonexistent Book", default=None)- assert result is None---class TestAutoSetRemoval:- """Test removal operations."""-- def test_remove_removes_by_key(self, llm_client, embedder):- """Test remove method by key."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- book = BookEntry(title="Remove This")- books.add(book)- assert len(books) == 1-- removed = books.remove("Remove This")- assert removed is not None- assert removed.title == "Remove This"- assert len(books) == 0- assert "Remove This" not in books-- def test_remove_nonexistent(self, llm_client, embedder):- """Test removing non-existent key returns None."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- result = books.remove("Nonexistent")- assert result is None---class TestAutoSetIteration:- """Test iteration operations."""-- def test_iter_over_items(self, llm_client, embedder):- """Test iterating over set items."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- titles = ["Book A", "Book B", "Book C"]- for title in titles:- books.add(BookEntry(title=title))-- collected = [b.title for b in books]- assert len(collected) == 3- assert set(collected) == set(titles)-- def test_len_operator(self, llm_client, embedder):- """Test len() operator."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- assert len(books) == 0-- for i in range(5):- books.add(BookEntry(title=f"Book_{i}"))-- assert len(books) == 5---class TestAutoSetFeedText:- """Test feed_text extraction."""-- def test_feed_text_populates_set(self, llm_client, embedder):- """Test feeding text extracts and deduplicates."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.feed_text("Some text about popular books and authors")-- # In mock mode, should have mock items- assert not books.empty()- assert len(books) > 0-- def test_feed_text_maintains_uniqueness(self, llm_client, embedder):- """Test that feed_text maintains deduplication."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- # Manually add some books- books.add(BookEntry(title="Python Guide"))- books.add(BookEntry(title="Java Basics"))- initial_size = len(books)-- # Feed text (in mock mode, generates mock data that might have same title)- books.feed_text("More information about programming books")-- # Size should be >= initial (due to mock data being indeterminate)- assert len(books) >= initial_size---class TestAutoSetRepr:- """Test string representations."""-- def test_repr(self, llm_client, embedder):- """Test __repr__ method."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- repr_str = repr(books)- assert "AutoSet" in repr_str- assert "BookEntry" in repr_str- assert "0" in repr_str-- def test_str(self, llm_client, embedder):- """Test __str__ method."""- books = AutoSet(- item_schema=BookEntry,- llm_client=llm_client,- embedder=embedder,- key_extractor=lambda x: x.title,- )-- books.add(BookEntry(title="Book One"))- books.add(BookEntry(title="Book Two"))-- str_repr = str(books)- assert "AutoSet" in str_repr- assert "2" in str_reprdiff --git a/tests/types/test_set_dedup.py b/tests/types/test_set_dedup.pynew file mode 100644index 0000000..4ba73ac--- /dev/null+++ b/tests/types/test_set_dedup.py@@ -0,0 +1,293 @@+"""Unit tests for AutoSet deduplication functionality."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoSet+from tests.fixtures import KeywordSchema+++class KeywordItemSchema(BaseModel):+ """Schema for keyword items."""++ term: str+ category: Optional[str] = None+ frequency: Optional[int] = None+++class TestAutoSetDedup:+ """Test cases for AutoSet deduplication functionality."""++ def test_init_and_empty_state(self, llm_client, embedder):+ """Test that AutoSet initializes correctly."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ assert auto_set.empty() is True+ assert len(auto_set) == 0++ def test_add_single_item(self, llm_client, embedder):+ """Test adding a single item."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python", category="Programming"))++ assert len(auto_set) == 1+ assert "Python" in auto_set++ def test_deduplication_by_key(self, llm_client, embedder):+ """Test that adding duplicate keys results in single item."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python", category="Programming"))+ auto_set.add(KeywordItemSchema(term="Python", category="Software"))++ assert len(auto_set) == 1+ assert "Python" in auto_set++ def test_multiple_unique_items(self, llm_client, embedder):+ """Test adding multiple unique items."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python", category="Programming"))+ auto_set.add(KeywordItemSchema(term="Java", category="Programming"))+ auto_set.add(KeywordItemSchema(term="React", category="Frontend"))++ assert len(auto_set) == 3++ def test_contains_true(self, llm_client, embedder):+ """Test that contains returns True for existing key."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ assert auto_set.contains("Python") is True+ assert "Python" in auto_set++ def test_contains_false(self, llm_client, embedder):+ """Test that contains returns False for non-existing key."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ assert auto_set.contains("Java") is False+ assert "Java" not in auto_set++ def test_get_existing_item(self, llm_client, embedder):+ """Test getting an existing item by key."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python", category="Programming"))++ item = auto_set.get("Python")++ assert item is not None+ assert item.term == "Python"+ assert item.category == "Programming"++ def test_get_nonexistent_item(self, llm_client, embedder):+ """Test getting a non-existent item returns None."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ item = auto_set.get("NotFound")++ assert item is None++ def test_get_with_default(self, llm_client, embedder):+ """Test getting with default value."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ result = auto_set.get("NotFound", default="default_value")++ assert result == "default_value"++ def test_remove_existing_item(self, llm_client, embedder):+ """Test removing an existing item."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))+ auto_set.add(KeywordItemSchema(term="Java"))++ removed = auto_set.remove("Python")++ assert len(auto_set) == 1+ assert "Python" not in auto_set+ assert removed.term == "Python"++ def test_remove_nonexistent_item(self, llm_client, embedder):+ """Test removing a non-existent item returns None."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ removed = auto_set.remove("NotFound")++ assert removed is None+ assert len(auto_set) == 1++ def test_discard_existing_item(self, llm_client, embedder):+ """Test discarding an existing item."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ auto_set.discard("Python")++ assert len(auto_set) == 0++ def test_discard_nonexistent_item(self, llm_client, embedder):+ """Test discarding a non-existent item doesn't raise."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ auto_set.discard("NotFound")++ assert len(auto_set) == 1++ def test_pop_item(self, llm_client, embedder):+ """Test popping an item."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))+ auto_set.add(KeywordItemSchema(term="Java"))++ popped = auto_set.pop()++ assert popped is not None+ assert len(auto_set) == 1++ def test_pop_from_empty_raises(self, llm_client, embedder):+ """Test that popping from empty set raises KeyError."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ with pytest.raises(KeyError):+ auto_set.pop()++ def test_keys_property(self, llm_client, embedder):+ """Test that keys property returns all unique keys."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))+ auto_set.add(KeywordItemSchema(term="Java"))++ keys = auto_set.keys++ assert len(keys) == 2+ assert "Python" in keys+ assert "Java" in keys++ def test_iteration(self, llm_client, embedder):+ """Test that AutoSet is iterable."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))+ auto_set.add(KeywordItemSchema(term="Java"))++ terms = [item.term for item in auto_set]++ assert len(terms) == 2+ assert "Python" in terms+ assert "Java" in terms++ def test_copy(self, llm_client, embedder):+ """Test copying the set."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ copied = auto_set.copy()++ assert len(copied) == 1+ assert "Python" in copied+ assert copied is not auto_setdiff --git a/tests/types/test_set_operations.py b/tests/types/test_set_operations.pynew file mode 100644index 0000000..07d79f9--- /dev/null+++ b/tests/types/test_set_operations.py@@ -0,0 +1,303 @@+"""Unit tests for AutoSet set operations."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoSet+++class KeywordItemSchema(BaseModel):+ """Schema for keyword items."""++ term: str+ category: Optional[str] = None+++class TestAutoSetOperations:+ """Test cases for AutoSet set operations (union, intersection, difference)."""++ def _create_set_with_terms(self, llm_client, embedder, terms):+ """Helper to create a set with given terms."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )+ for term in terms:+ auto_set.add(KeywordItemSchema(term=term))+ return auto_set++ def test_union_operator(self, llm_client, embedder):+ """Test union operator |."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1 | set2++ assert len(result) == 3+ assert "Python" in result+ assert "Java" in result+ assert "React" in result++ def test_union_named_method(self, llm_client, embedder):+ """Test union named method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1.union(set2)++ assert len(result) == 3++ def test_union_preserves_original(self, llm_client, embedder):+ """Test that union preserves original sets."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ _ = set1 | set2++ assert len(set1) == 1+ assert len(set2) == 1++ def test_intersection_operator(self, llm_client, embedder):+ """Test intersection operator &."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React", "Vue"])++ result = set1 & set2++ assert len(result) == 2+ assert "Java" in result+ assert "React" in result++ def test_intersection_named_method(self, llm_client, embedder):+ """Test intersection named method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1.intersection(set2)++ assert len(result) == 1+ assert "Java" in result++ def test_intersection_no_common(self, llm_client, embedder):+ """Test intersection with no common elements."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ result = set1 & set2++ assert len(result) == 0++ def test_difference_operator(self, llm_client, embedder):+ """Test difference operator -."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1 - set2++ assert len(result) == 1+ assert "Python" in result+ assert "Java" not in result++ def test_difference_named_method(self, llm_client, embedder):+ """Test difference named method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ result = set1.difference(set2)++ assert len(result) == 1+ assert "Python" in result++ def test_symmetric_difference_operator(self, llm_client, embedder):+ """Test symmetric difference operator ^."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1 ^ set2++ assert len(result) == 2+ assert "Python" in result+ assert "React" in result+ assert "Java" not in result++ def test_symmetric_difference_named_method(self, llm_client, embedder):+ """Test symmetric difference named method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ result = set1.symmetric_difference(set2)++ assert len(result) == 2++ def test_symmetric_difference_with_no_overlap(self, llm_client, embedder):+ """Test symmetric difference with no overlapping elements."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ result = set1 ^ set2++ assert len(result) == 2+ assert "Python" in result+ assert "Java" in result+++class TestAutoSetComparison:+ """Test cases for AutoSet comparison operators."""++ def _create_set_with_terms(self, llm_client, embedder, terms):+ """Helper to create a set with given terms."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )+ for term in terms:+ auto_set.add(KeywordItemSchema(term=term))+ return auto_set++ def test_equality_same_keys(self, llm_client, embedder):+ """Test equality with same keys."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 == set2++ def test_equality_different_keys(self, llm_client, embedder):+ """Test equality with different keys."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ assert set1 != set2++ def test_equality_different_lengths(self, llm_client, embedder):+ """Test equality with different lengths."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python"])++ assert set1 != set2++ def test_subset_operator(self, llm_client, embedder):+ """Test subset operator <=."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 <= set2++ def test_subset_not_equal(self, llm_client, embedder):+ """Test subset with equal sets."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 <= set2++ def test_subset_not_reflexive_superset(self, llm_client, embedder):+ """Test that subset is not true when superset is smaller."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert not (set1 <= set2)++ def test_proper_subset_operator(self, llm_client, embedder):+ """Test proper subset operator <."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 < set2++ def test_proper_subset_equal_sets(self, llm_client, embedder):+ """Test that proper subset is false for equal sets."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert not (set1 < set2)++ def test_superset_operator(self, llm_client, embedder):+ """Test superset operator >=."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 >= set2++ def test_proper_superset_operator(self, llm_client, embedder):+ """Test proper superset operator >."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1 > set2++ def test_proper_superset_equal_sets(self, llm_client, embedder):+ """Test that proper superset is false for equal sets."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert not (set1 > set2)++ def test_issubset_method(self, llm_client, embedder):+ """Test issubset method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1.issubset(set2)++ def test_issuperset_method(self, llm_client, embedder):+ """Test issuperset method."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java", "React"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])++ assert set1.issuperset(set2)++ def test_isdisjoint_true(self, llm_client, embedder):+ """Test isdisjoint when sets have no common elements."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java"])++ assert set1.isdisjoint(set2)++ def test_isdisjoint_false(self, llm_client, embedder):+ """Test isdisjoint when sets have common elements."""+ set1 = self._create_set_with_terms(llm_client, embedder, ["Python", "Java"])+ set2 = self._create_set_with_terms(llm_client, embedder, ["Java", "React"])++ assert not set1.isdisjoint(set2)+++class TestAutoSetRepr:+ """Test cases for AutoSet string representation."""++ def test_repr(self, llm_client, embedder):+ """Test __repr__ method."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))+ auto_set.add(KeywordItemSchema(term="Java"))++ repr_str = repr(auto_set)++ assert "AutoSet" in repr_str+ assert "KeywordItemSchema" in repr_str+ assert "2" in repr_str++ def test_str(self, llm_client, embedder):+ """Test __str__ method."""+ auto_set = AutoSet(+ item_schema=KeywordItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.term,+ )++ auto_set.add(KeywordItemSchema(term="Python"))++ str_repr = str(auto_set)++ assert "AutoSet" in str_repr+ assert "1" in str_reprdiff --git a/tests/types/test_set_operators.py b/tests/types/test_set_operators.pynew file mode 100644index 0000000..c196f06--- /dev/null+++ b/tests/types/test_set_operators.py@@ -0,0 +1,190 @@+"""Unit tests for AutoSet key operations."""++import pytest+from pydantic import BaseModel, Field+from typing import Optional++from hyperextract.types import AutoSet+++class ProductItemSchema(BaseModel):+ """Schema for product items."""++ sku: str+ name: str+ price: Optional[float] = None+++class TestAutoSetOperators:+ """Test cases for AutoSet operator overloads."""++ def test_len(self, llm_client, embedder):+ """Test len() function."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ assert len(auto_set) == 0++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))+ auto_set.add(ProductItemSchema(sku="P002", name="Product 2"))++ assert len(auto_set) == 2++ def test_contains_with_key(self, llm_client, embedder):+ """Test __contains__ with key value."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))++ assert "P001" in auto_set+ assert "P002" not in auto_set++ def test_iter(self, llm_client, embedder):+ """Test __iter__."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))+ auto_set.add(ProductItemSchema(sku="P002", name="Product 2"))++ items = list(auto_set)++ assert len(items) == 2+ assert all(isinstance(item, ProductItemSchema) for item in items)++ def test_union_operator(self, llm_client, embedder):+ """Test that | (union) operator is supported for AutoSet."""+ auto_set1 = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )+ auto_set2 = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set1.add(ProductItemSchema(sku="P001", name="Product 1"))+ auto_set2.add(ProductItemSchema(sku="P002", name="Product 2"))++ result = auto_set1 | auto_set2++ assert len(result) == 2+ assert "P001" in result+ assert "P002" in result+++class TestAutoSetKeyOperations:+ """Test cases for AutoSet key-based operations."""++ def test_key_extractor(self, llm_client, embedder):+ """Test that key extractor is used correctly."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set.add(ProductItemSchema(sku="ABC123", name="Test Product"))++ assert "ABC123" in auto_set+ assert auto_set.get("ABC123").name == "Test Product"++ def test_composite_key_extractor(self, llm_client, embedder):+ """Test composite key extractor."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: f"{x.sku}_{x.name}",+ )++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))++ assert "P001_Product 1" in auto_set++ def test_batch_add_via_memory(self, llm_client, embedder):+ """Test batch adding via internal memory."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ items = [+ ProductItemSchema(sku="P001", name="Product 1"),+ ProductItemSchema(sku="P002", name="Product 2"),+ ]++ auto_set._data_memory.add(items)++ assert len(auto_set) == 2+ assert "P001" in auto_set+ assert "P002" in auto_set+++class TestAutoSetStateManagement:+ """Test cases for AutoSet state management."""++ def test_clear(self, llm_client, embedder):+ """Test clearing the set."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))+ auto_set.add(ProductItemSchema(sku="P002", name="Product 2"))++ assert len(auto_set) == 2++ auto_set.clear()++ assert len(auto_set) == 0+ assert auto_set.empty() is True++ def test_empty_after_clear(self, llm_client, embedder):+ """Test empty() after clear()."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ auto_set.add(ProductItemSchema(sku="P001", name="Product 1"))+ auto_set.clear()++ assert auto_set.empty() is True++ def test_metadata_initialized(self, llm_client, embedder):+ """Test that metadata is initialized correctly."""+ auto_set = AutoSet(+ item_schema=ProductItemSchema,+ llm_client=llm_client,+ embedder=embedder,+ key_extractor=lambda x: x.sku,+ )++ assert "created_at" in auto_set.metadata+ assert "updated_at" in auto_set.metadatadiff --git a/tests/types/test_spatial_graph.py b/tests/types/test_spatial_graph.pydeleted file mode 100644index 752b619..0000000--- a/tests/types/test_spatial_graph.py+++ /dev/null@@ -1,235 +0,0 @@-"""Tests for AutoSpatialGraph - spatial graph extraction with location-aware deduplication."""--import pytest-from pydantic import BaseModel, Field-from typing import Optional, Tuple-from ontomem.merger import MergeStrategy-from hyperextract.types.spatial_graph import AutoSpatialGraph---class Entity(BaseModel):- """Test schema for spatial graph nodes."""-- name: str = Field(default="", description="Entity name")---class SpatialEdge(BaseModel):- """Test schema for spatial edges."""-- src: str = Field(default="", description="Source entity")- dst: str = Field(default="", description="Target entity")- rel: str = Field(default="", description="Relationship type")- place: Optional[str] = Field(default=None, description="Location of relationship")---class TestAutoSpatialGraphBasics:- """Test basic AutoSpatialGraph functionality."""-- def test_initialization_with_default_location(self, llm_client, embedder):- """Test AutoSpatialGraph initialization with default observation_location."""- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Should have default observation_location- assert kg.observation_location is not None- assert kg.observation_location == "Unknown Location"-- def test_initialization_with_custom_location(self, llm_client, embedder):- """Test AutoSpatialGraph initialization with custom observation_location."""- custom_location = "Paris, France"-- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_location=custom_location,- )-- assert kg.observation_location == custom_location---class TestAutoSpatialGraphLocationInjection:- """Test that observation_location is properly injected into prompts."""-- def test_observation_location_in_edge_prompt(self, llm_client, embedder):- """Test that observation_location is included in edge extraction prompt."""- observation_location = "Tokyo"-- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_location=observation_location,- )-- # The observation_location should be stored- assert kg.observation_location == observation_location- # And should be in prompts- assert "location" in kg.edge_prompt.lower() or "place" in kg.edge_prompt.lower()-- def test_observation_location_in_graph_prompt(self, llm_client, embedder):- """Test that observation_location is in the full graph extraction prompt."""- observation_location = "New York City"-- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_location=observation_location,- )-- assert kg.observation_location == observation_location---class TestAutoSpatialGraphKeyGeneration:- """Test spatial edge key generation."""-- def test_edge_key_includes_location(self, llm_client, embedder):- """Test that edge keys differentiate based on location."""- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # The internal edge_key_extractor should combine base key and location- edge1 = SpatialEdge(src="Shop", dst="Customer", rel="at", place="London")- edge2 = SpatialEdge(src="Shop", dst="Customer", rel="at", place="Tokyo")- edge3 = SpatialEdge(src="Shop", dst="Customer", rel="at")-- # Adding edges with different locations should create different keys- kg._edge_memory.add([edge1, edge2, edge3])-- # Should have at least 2 unique edges (different locations)- assert kg._edge_memory.size >= 2---class TestAutoSpatialGraphExtraction:- """Test spatial graph extraction."""-- def test_feed_text_two_stage(self, llm_client, embedder):- """Test feeding text in two-stage extraction mode."""- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- kg.feed_text("The library is located next to the museum. They are in downtown.")-- assert not kg.empty()- assert len(kg.data.nodes) > 0-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test extract() creates new instance."""- kg1 = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- kg2 = kg1.parse("""Company headquarters in San Francisco connects to regional centers:- New York for East Coast operations, Chicago for Midwest, and Denver for Mountain region.- Each regional center manages local office branches.""")-- assert kg1 is not kg2- assert kg1.empty() is True- assert kg2.empty() is False---class TestAutoSpatialGraphPrompts:- """Test spatial prompt construction."""-- def test_node_prompt_excludes_locations(self, llm_client, embedder):- """Test that node extraction prompt instructs NOT to extract locations."""- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Node prompt should warn not to extract locations- assert "location" in kg.node_prompt.lower() or "spatial" in kg.node_prompt.lower()-- def test_edge_prompt_includes_spatial_rules(self, llm_client, embedder):- """Test that edge prompt includes spatial resolution rules."""- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Edge prompt should contain spatial/location rules- assert "location" in kg.edge_prompt.lower() or "observation" in kg.edge_prompt.lower()---class TestAutoSpatialGraphLocationContext:- """Test location context handling."""-- def test_custom_location_context(self, llm_client, embedder):- """Test that custom location context is used."""- locations = ["Office A", "Conference Room B", "Parking Lot"]-- for location in locations:- kg = AutoSpatialGraph(- node_schema=Entity,- edge_schema=SpatialEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_location=location,- )-- assert kg.observation_location == locationdiff --git a/tests/types/test_spatio_temporal_graph.py b/tests/types/test_spatio_temporal_graph.pydeleted file mode 100644index 641db8e..0000000--- a/tests/types/test_spatio_temporal_graph.py+++ /dev/null@@ -1,288 +0,0 @@-"""Tests for AutoSpatioTemporalGraph - spatio-temporal graph extraction."""--import pytest-from pydantic import BaseModel, Field-from typing import Optional, Tuple-from ontomem.merger import MergeStrategy-from hyperextract.types.spatio_temporal_graph import AutoSpatioTemporalGraph---class Entity(BaseModel):- """Test schema for spatio-temporal graph nodes."""-- name: str = Field(default="", description="Entity name")---class STEdge(BaseModel):- """Test schema for spatio-temporal edges."""-- src: str = Field(default="", description="Source entity")- dst: str = Field(default="", description="Target entity")- rel: str = Field(default="", description="Relationship type")- time: Optional[str] = Field(default=None, description="Time of relationship")- place: Optional[str] = Field(default=None, description="Location of relationship")---class TestAutoSpatioTemporalGraphBasics:- """Test basic AutoSpatioTemporalGraph functionality."""-- def test_initialization_with_defaults(self, llm_client, embedder):- """Test AutoSpatioTemporalGraph initialization with defaults."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Should have default observation_time and observation_location- assert kg.observation_time is not None- assert kg.observation_location is not None-- def test_initialization_with_custom_context(self, llm_client, embedder):- """Test AutoSpatioTemporalGraph with custom time and location."""- custom_time = "2024-06-15"- custom_location = "Singapore"-- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_time=custom_time,- observation_location=custom_location,- )-- assert kg.observation_time == custom_time- assert kg.observation_location == custom_location---class TestAutoSpatioTemporalGraphContextInjection:- """Test that both time and location contexts are injected."""-- def test_both_contexts_in_prompts(self, llm_client, embedder):- """Test that both observation_time and observation_location are in prompts."""- custom_time = "2024-01-01"- custom_location = "Berlin"-- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_time=custom_time,- observation_location=custom_location,- )-- # Both contexts should be stored- assert kg.observation_time == custom_time- assert kg.observation_location == custom_location- # Prompts should reference spatio-temporal concepts- assert ("time" in kg.edge_prompt.lower() and "location" in kg.edge_prompt.lower()) \- or "spatio" in kg.edge_prompt.lower()---class TestAutoSpatioTemporalGraphKeyGeneration:- """Test composite key generation combining time and location."""-- def test_edge_key_combines_time_and_location(self, llm_client, embedder):- """Test that edge keys combine both time and location."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Create edges: same relation at different times and places- e1 = STEdge(src="Alice", dst="Bob", rel="meet", time="2023-06-15", place="London")- e2 = STEdge(src="Alice", dst="Bob", rel="meet", time="2023-06-15", place="Paris")- e3 = STEdge(src="Alice", dst="Bob", rel="meet", time="2023-07-15", place="London")- e4 = STEdge(src="Alice", dst="Bob", rel="meet", time="2023-07-15", place="Paris")- e5 = STEdge(src="Alice", dst="Bob", rel="meet") # No time or place-- kg._edge_memory.add([e1, e2, e3, e4, e5])-- # All 5 should be unique (different combinations of time and location)- assert kg._edge_memory.size >= 3 # At least 3 different combinations---class TestAutoSpatioTemporalGraphExtraction:- """Test spatio-temporal graph extraction."""-- def test_feed_text_two_stage(self, llm_client, embedder):- """Test feeding text in two-stage extraction mode."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- kg.feed_text("In 2023, Alice and Bob met at the conference in Barcelona.")-- assert not kg.empty()- assert len(kg.data.nodes) > 0-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test extract() creates new instance."""- kg1 = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- kg2 = kg1.parse("""Tech conference tour 2024: January workshop in San Francisco,- March seminar in New York, June training in London, September workshop in Tokyo.- Each event has 200+ attendees sharing innovations and networking.""")-- assert kg1 is not kg2- assert kg1.empty() is True- assert kg2.empty() is False---class TestAutoSpatioTemporalGraphPrompts:- """Test spatio-temporal prompt construction."""-- def test_edge_prompt_includes_temporal_rules(self, llm_client, embedder):- """Test that edge prompt includes temporal extraction rules."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Edge prompt should contain both temporal and spatial rules- edge_prompt_lower = kg.edge_prompt.lower()- assert "time" in edge_prompt_lower or "date" in edge_prompt_lower or "observation" in edge_prompt_lower-- def test_edge_prompt_includes_spatial_rules(self, llm_client, embedder):- """Test that edge prompt includes spatial extraction rules."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Edge prompt should contain spatial rules- edge_prompt_lower = kg.edge_prompt.lower()- assert "location" in edge_prompt_lower or "place" in edge_prompt_lower or "observation" in edge_prompt_lower-- def test_node_prompt_excludes_temporal_spatial_markers(self, llm_client, embedder):- """Test that node extraction prompt excludes dates and locations."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Node prompt should warn against extracting dates/times and locations- node_prompt_lower = kg.node_prompt.lower()- assert "exclude" in node_prompt_lower or "never" in node_prompt_lower or "not" in node_prompt_lower---class TestAutoSpatioTemporalGraphConsistency:- """Test spatio-temporal graph consistency."""-- def test_all_dimensions_in_edge(self, llm_client, embedder):- """Test that edges can have all spatio-temporal dimensions."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Add nodes- kg._node_memory.add([Entity(name="Alice"), Entity(name="Bob")])-- # Edge with all dimensions- full_edge = STEdge(src="Alice", dst="Bob", rel="meet", time="2024-06-15", place="Summit")- kg._edge_memory.add([full_edge])-- assert kg._edge_memory.size > 0- edge_data = kg._edge_memory.items[0]- assert edge_data.time == "2024-06-15"- assert edge_data.place == "Summit"-- def test_partial_dimensions(self, llm_client, embedder):- """Test edges with partial spatio-temporal information."""- kg = AutoSpatioTemporalGraph(- node_schema=Entity,- edge_schema=STEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.time or "",- location_in_edge_extractor=lambda x: x.place or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Add nodes- kg._node_memory.add([Entity(name="Alice"), Entity(name="Bob")])-- # Edges with partial information- time_only = STEdge(src="Alice", dst="Bob", rel="call", time="2024-06-15")- place_only = STEdge(src="Alice", dst="Bob", rel="visit", place="Office")- neither = STEdge(src="Alice", dst="Bob", rel="know")-- kg._edge_memory.add([time_only, place_only, neither])-- # All should be added (different combinations)- assert kg._edge_memory.size >= 2diff --git a/tests/types/test_temporal_graph.py b/tests/types/test_temporal_graph.pydeleted file mode 100644index 7e1bc93..0000000--- a/tests/types/test_temporal_graph.py+++ /dev/null@@ -1,243 +0,0 @@-"""Tests for AutoTemporalGraph - temporal graph extraction with time-aware deduplication."""--import pytest-from pydantic import BaseModel, Field-from typing import Optional, Tuple-from datetime import datetime, timedelta-from ontomem.merger import MergeStrategy-from hyperextract.types.temporal_graph import AutoTemporalGraph---class Entity(BaseModel):- """Test schema for temporal graph nodes."""-- name: str = Field(default="", description="Entity name")---class TemporalEdge(BaseModel):- """Test schema for temporal edges."""-- src: str = Field(default="", description="Source entity")- dst: str = Field(default="", description="Target entity")- rel: str = Field(default="", description="Relationship type")- year: Optional[str] = Field(default=None, description="Year of relationship")---class TestAutoTemporalGraphBasics:- """Test basic AutoTemporalGraph functionality."""-- def test_initialization_with_default_time(self, llm_client, embedder):- """Test AutoTemporalGraph initialization with default observation_time."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Should have default observation_time (today)- assert kg.observation_time is not None- assert len(kg.observation_time) == 10 # YYYY-MM-DD format-- def test_initialization_with_custom_time(self, llm_client, embedder):- """Test AutoTemporalGraph initialization with custom observation_time."""- custom_time = "2024-06-15"-- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_time=custom_time,- )-- assert kg.observation_time == custom_time---class TestAutoTemporalGraphTimeInjection:- """Test that observation_time is properly injected into prompts."""-- def test_observation_time_in_edge_prompt(self, llm_client, embedder):- """Test that observation_time is included in edge extraction prompt."""- observation_time = "2023-01-01"-- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_time=observation_time,- )-- # The observation_time should be in the prompt template- assert observation_time == kg.observation_time- assert "{observation_time}" in kg.edge_prompt or observation_time in kg.prompt-- def test_observation_time_in_graph_prompt(self, llm_client, embedder):- """Test that observation_time is in the full graph extraction prompt."""- observation_time = "2024-12-25"-- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- observation_time=observation_time,- )-- assert observation_time == kg.observation_time---class TestAutoTemporalGraphKeyGeneration:- """Test temporal edge key generation."""-- def test_edge_key_includes_time(self, llm_client, embedder):- """Test that edge keys differentiate based on time."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # The internal edge_key_extractor should combine base key and time- # Testing the logic of what the actual implementation does-- edge1 = TemporalEdge(src="Alice", dst="Bob", rel="friend", year="2020")- edge2 = TemporalEdge(src="Alice", dst="Bob", rel="friend", year="2021")- edge3 = TemporalEdge(src="Alice", dst="Bob", rel="friend")-- # Due to the composite key logic, edges with different years should have different keys- # And edge without time should be different from edges with time-- # We can test this by adding them to the edge memory- kg._edge_memory.add([edge1, edge2, edge3])-- # Should have 3 unique edges (all different times)- assert kg._edge_memory.size >= 2 # At least different from no-time case---class TestAutoTemporalGraphExtraction:- """Test temporal graph extraction."""-- def test_feed_text_two_stage(self, llm_client, embedder):- """Test feeding text in two-stage extraction mode."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- extraction_mode="two_stage",- )-- kg.feed_text("In 2020, Alice collaborated with Bob on a research project.")-- assert not kg.empty()- assert len(kg.data.nodes) > 0-- def test_extract_creates_new_instance(self, llm_client, embedder):- """Test extract() creates new instance."""- kg1 = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- kg2 = kg1.parse("""ProjectX timeline: Q1 2023 requirements gathering with stakeholders,- Q2 2023 architecture design phase, Q3 2023 development sprint cycles,- Q4 2023 beta testing and Q1 2024 production launch.""")-- assert kg1 is not kg2- assert kg1.empty() is True- assert kg2.empty() is False---class TestAutoTemporalGraphPrompts:- """Test temporal prompt construction."""-- def test_node_prompt_excludes_dates(self, llm_client, embedder):- """Test that node extraction prompt instructs NOT to extract dates."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Node prompt should warn not to extract dates- assert "date" in kg.node_prompt.lower() or "time" in kg.node_prompt.lower()-- def test_edge_prompt_includes_relative_time_rules(self, llm_client, embedder):- """Test that edge prompt includes relative time resolution rules."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Edge prompt should contain rules about relative time- assert "relative time" in kg.edge_prompt.lower() or "observation" in kg.edge_prompt.lower()---class TestAutoTemporalGraphConsistency:- """Test consistency between nodes and edges."""-- def test_temporal_edge_participants_are_validated(self, llm_client, embedder):- """Test that temporal edges reference existing nodes."""- kg = AutoTemporalGraph(- node_schema=Entity,- edge_schema=TemporalEdge,- node_key_extractor=lambda x: x.name,- edge_key_extractor=lambda x: f"{x.src}-{x.rel}-{x.dst}",- time_in_edge_extractor=lambda x: x.year or "",- nodes_in_edge_extractor=lambda x: (x.src, x.dst),- llm_client=llm_client,- embedder=embedder,- )-- # Create nodes- kg._node_memory.add([Entity(name="Alice"), Entity(name="Bob")])-- # Create valid edges (participants exist)- valid_edge = TemporalEdge(src="Alice", dst="Bob", rel="knows", year="2022")- kg._edge_memory.add([valid_edge])-- assert kg._edge_memory.size > 0