Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: 更新文档模板和配置文件,优化CLI交互体验
yifanfeng97302ceff4268d
diff --git a/README.md b/README.mdindex 8b2b908..f1c2872 100644--- a/README.md+++ b/README.md@@ -6,157 +6,85 @@Transform unstructured documents into **searchable, visual, structured knowledge** — with just one command.-[📖 English Version](#) · [中文版](./README_ZH.md)+[📖 English Version](./README.md) · [中文版](./README_ZH.md)------## ✨ Features--| Feature | Description |-|---------|-------------|-| ⚡ **CLI-First** | One command to extract knowledge from any document |-| 🎯 **8 Structures** | Knowledge Graph, Timeline, Spatial, Hypergraph... |-| 👁️ **Visual** | Interactive visualization with OntoSight |-| 🔍 **Searchable** | Semantic search across all your knowledge |-| 🌐 **Bilingual** | Full support for English and Chinese |-| 📦 **200+ Templates** | Ready-to-use domain-specific templates |-------## ❌ Before | ✅ After--| Before | After |-|--------|-------|-| Walls of text | **Clear visual structure** |-| ❌ Hours of reading | ✅ **Instant clarity** |-| ❌ Hard to find key info | ✅ **Semantic search** |-| ❌ Can't compare docs | ✅ **Structured comparison** |-| ❌ Scattered insights | ✅ **Knowledge accumulation** |--+---## ⚡ Quick Start-### Installation-```bashpip install hyper-extract-```--### Usage--```bash-# Extract structure from document-he parse document.pdf -o kb-# Visualize the knowledge+he parse document.md -o kbhe show kb--# Semantic searchhe search kb "key insights"--# Interactive Q&A-he talk kb -i```----## 🧩 8 Knowledge Structures+## 🧩 Knowledge Structures-**8 different structures** for different needs:+-| Type | Icon | Best For | Example |-|------|------|----------|---------|-| **AutoModel** | 📋 | Structured reports | Financial statements |-| **AutoList** | 📝 | Key points | Meeting notes |-| **AutoSet** | 📦 | Entity registry | Product catalog |-| **AutoGraph** | 🔗 | Relations | Social networks |-| **AutoTemporalGraph** | ⏱️ | Event sequences | News timeline |-| **AutoSpatialGraph** | 📍 | Locations | Delivery routes |-| **AutoSpatioTemporalGraph** | 🌏 | Time + Space | Historical events |-| **AutoHypergraph** | 🌐 | Complex relations | Legal cases |+8 structures for extracting different types of information:------+**Scalar Types:**+| Structure | Best For | Example |+|-----------|----------|---------|+| AutoModel | Structured reports | Financial statements |+| AutoList | Key points | Meeting notes |+| AutoSet | Entity collection | Product catalog |-## 🎯 Use Cases+**Graph Types:**+| Structure | Best For | Example |+|-----------|----------|---------|+| AutoGraph | Binary relations | Social networks |+| AutoHypergraph | Multi-party events | Legal disputes |+| AutoTemporalGraph | Event sequences | News timeline |+| AutoSpatialGraph | Locations | Delivery routes |+| AutoSpatioTemporalGraph | Events in time & space | Historical battles |-| Domain | What You Get | Example |-|--------|--------------|---------|-| 📊 **Finance** | Extract insights from earnings reports | `he parse report.pdf -o kb -l en` |-| ⚖️ **Legal** | Structure contracts, case laws | `he parse contract.pdf -o kb -t hypergraph` |-| 🏥 **Medical** | Organize patient histories | `he parse records.pdf -o kb` |-| 📚 **Research** | Extract key findings from papers | `he parse paper.pdf -o kb` |-| 📋 **Meetings** | Transform notes into timelines | `he parse notes.md -o kb` |+### Comparison with Other Libraries-+| Feature | KG-Gen | ATOM | Graphiti | LightRAG | Hyper-Extract |+|---------|--------|------|----------|----------|---------------|+| Knowledge Graph | ✅ | ✅ | ❌ | ✅ | ✅ |+| Temporal Graph | ❌ | ✅ | ✅ | ❌ | ✅ |+| Spatial Graph | ❌ | ❌ | ❌ | ❌ | ✅ |+| Hypergraph | ❌ | ❌ | ❌ | ❌ | ✅ |+| Templates | ❌ | ❌ | ❌ | ❌ | ✅ |+| CLI Tool | ❌ | ❌ | ❌ | ❌ | ✅ |----## 🔧 Architecture+## 🌍 Domain Templates-<details>-<summary><strong>Technical Details (Click to Expand)</strong></summary>+-```-hyper-extract/-├── cli/ # 💻 CLI Interface (he command)-│ ├── commands/ # parse, talk, search, show...-│ └── __main__.py # Entry point-│-├── types/ # 🧩 8 Knowledge Structures-│ ├── model.py # AutoModel-│ ├── list.py # AutoList-│ ├── set.py # AutoSet-│ ├── graph.py # AutoGraph-│ ├── hypergraph.py # AutoHypergraph-│ ├── temporal_graph.py # AutoTemporalGraph-│ ├── spatial_graph.py # AutoSpatialGraph-│ └── spatio_temporal_graph.py-│-├── methods/ # 🔬 Extraction Engines-│ ├── rag/ # LightRAG, HyperRAG, CogRAG-│ └── typical/ # KG-Gen, ATOM-│-└── templates/ # 🌍 200+ Domain Templates- ├── zh/ # Chinese templates- └── en/ # English templates-```+-### Supported Methods+Pre-built templates for common domains:-| Method | Graph | Temporal | Spatial | Hypergraph |-|--------|-------|----------|---------|------------|-| KG-Gen | ✅ | ❌ | ❌ | ❌ |-| ATOM | ✅ | ✅ | ❌ | ❌ |-| Graphiti | ❌ | ✅ | ❌ | ❌ |-| LightRAG | ✅ | ❌ | ❌ | ❌ |-| Hyper-RAG | ❌ | ❌ | ❌ | ✅ |-| **Hyper-Extract** | ✅ | ✅ | ✅ | ✅ |+| Domain | Templates |+|--------|-----------|+| Finance | Earnings reports, shareholder structure, risk factors |+| Legal | Contracts, case law, compliance requirements |+| Medicine | Clinical records, pharmacology, treatment plans |+| Industry | Equipment specs, incident reports, safety procedures |+| General | Meeting notes, articles, research papers |+| TCM | Herbal formulas, meridian flow, syndrome reasoning |-</details>+See [Template Gallery](hyperextract/templates/) for all templates.----## 📚 Documentation & Resources+## 📚 Documentation- [📖 Full Documentation](docs/)- [💻 Examples](examples/)- [🏷️ Template Gallery](hyperextract/templates/)------## 🤝 Contributing & Support--Welcome! Please feel free to submit issues and pull requests.--If you find this project helpful, please give us a ⭐ to show your support!-------*Built with ❤️ for the AI Community*diff --git a/README_ZH.md b/README_ZH.mdindex 137326f..af1590e 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -8,155 +8,83 @@[📖 English Version](./README.md) · [中文版](#)------## ✨ 核心特性--| 特性 | 说明 |-|------|------|-| ⚡ **CLI 优先** | 一条命令提取任何文档的知识 |-| 🎯 **8 种结构** | 知识图谱、时间线、空间图、超图... |-| 👁️ **可视化** | 使用 OntoSight 进行交互式可视化 |-| 🔍 **可搜索** | 语义搜索所有知识 |-| 🌐 **双语支持** | 完整支持中文和英文 |-| 📦 **200+ 模板** | 开箱即用的领域模板 |-------## ❌ 以前 | ✅ 现在--| 以前 | 现在 |-|------|------|-| 密密麻麻的文字 | **清晰的结构** |-| ❌ 花几小时阅读 | ✅ **即时清晰** |-| ❌ 找不到关键信息 | ✅ **语义搜索** |-| ❌ 无法对比文档 | ✅ **结构化对比** |-| ❌ 洞察碎片化 | ✅ **知识沉淀** |--+---## ⚡ 快速开始-### 安装-```bashpip install hyper-extract-```-### 使用--```bash-# 从文档提取结构-he parse document.pdf -o kb--# 可视化知识+he parse document.md -o kbhe show kb--# 语义搜索he search kb "关键信息"--# 交互式问答-he talk kb -i```----## 🧩 8 种知识结构--**8 种不同结构**,满足不同需求:--| 类型 | 图标 | 适用场景 | 示例 |-|------|------|----------|------|-| **AutoModel** | 📋 | 结构化报告 | 财务报表 |-| **AutoList** | 📝 | 要点提取 | 会议记录 |-| **AutoSet** | 📦 | 实体注册 | 产品目录 |-| **AutoGraph** | 🔗 | 关系分析 | 社交网络 |-| **AutoTemporalGraph** | ⏱️ | 事件序列 | 新闻时间线 |-| **AutoSpatialGraph** | 📍 | 地理位置 | 配送路线 |-| **AutoSpatioTemporalGraph** | 🌏 | 时间+空间 | 历史事件 |-| **AutoHypergraph** | 🌐 | 复杂关系 | 法律案件 |--+## 🧩 知识结构----+-## 🎯 使用场景+8 种知识结构,满足不同文档类型:-| 领域 | 你将获得 | 示例 |+**标量类型:**+| 结构 | 适用场景 | 示例 ||------|----------|------|-| 📊 **财务** | 从财报中提取洞察 | `he parse report.pdf -o kb -l en` |-| ⚖️ **法律** | 结构化合同、法条 | `he parse contract.pdf -o kb -t hypergraph` |-| 🏥 **医疗** | 整理病历、治疗方案 | `he parse records.pdf -o kb` |-| 📚 **研究** | 从论文中提取关键发现 | `he parse paper.pdf -o kb` |-| 📋 **会议** | 转化为可操作的洞察 | `he parse notes.md -o kb` |+| AutoModel | 结构化报告 | 财务报表 |+| AutoList | 要点列表 | 会议记录 |+| AutoSet | 实体集合 | 产品目录 |-+**图类型:**+| 结构 | 适用场景 | 示例 |+|------|----------|------|+| AutoGraph | 二元关系 | 社交网络 |+| AutoHypergraph | 多方事件 | 法律纠纷 |+| AutoTemporalGraph | 事件序列 | 新闻时间线 |+| AutoSpatialGraph | 地理位置 | 配送路线 |+| AutoSpatioTemporalGraph | 时空事件 | 历史战役 |++### 与其他库的对比++| 功能 | KG-Gen | ATOM | Graphiti | LightRAG | Hyper-Extract |+|------|--------|------|----------|----------|---------------|+| 知识图谱 | ✅ | ✅ | ❌ | ✅ | ✅ |+| 时序图谱 | ❌ | ✅ | ✅ | ❌ | ✅ |+| 空间图谱 | ❌ | ❌ | ❌ | ❌ | ✅ |+| 超图 | ❌ | ❌ | ❌ | ❌ | ✅ |+| 模板 | ❌ | ❌ | ❌ | ❌ | ✅ |+| CLI 工具 | ❌ | ❌ | ❌ | ❌ | ✅ |----## 🔧 架构+## 🌍 领域模板-<details>-<summary><strong>技术细节(点击展开)</strong></summary>+-```-hyper-extract/-├── cli/ # 💻 CLI 接口 (he 命令)-│ ├── commands/ # parse, talk, search, show...-│ └── __main__.py # 入口点-│-├── types/ # 🧩 8 种知识结构-│ ├── model.py # AutoModel-│ ├── list.py # AutoList-│ ├── set.py # AutoSet-│ ├── graph.py # AutoGraph-│ ├── hypergraph.py # AutoHypergraph-│ ├── temporal_graph.py # AutoTemporalGraph-│ ├── spatial_graph.py # AutoSpatialGraph-│ └── spatio_temporal_graph.py-│-├── methods/ # 🔬 提取引擎-│ ├── rag/ # LightRAG, HyperRAG, CogRAG-│ └── typical/ # KG-Gen, ATOM-│-└── templates/ # 🌍 200+ 领域模板- ├── zh/ # 中文模板- └── en/ # 英文模板-```+-### 支持的方法+开箱即用的领域模板:-| 方法 | 图谱 | 时序 | 空间 | 超图 |-|------|------|------|------|------|-| KG-Gen | ✅ | ❌ | ❌ | ❌ |-| ATOM | ✅ | ✅ | ❌ | ❌ |-| Graphiti | ❌ | ✅ | ❌ | ❌ |-| LightRAG | ✅ | ❌ | ❌ | ❌ |-| Hyper-RAG | ❌ | ❌ | ❌ | ✅ |-| **Hyper-Extract** | ✅ | ✅ | ✅ | ✅ |+| 领域 | 模板 |+|------|------|+| 金融 | 财报、股权结构、风险因子 |+| 法律 | 合同、法条、合规要求 |+| 医疗 | 临床记录、药理学、治疗方案 |+| 工业 | 设备规格、事故报告、安全规程 |+| 通用 | 会议记录、文章、研究论文 |+| 中医 | 中药方剂、经络流转、证候推理 |-</details>+查看 [模板库](hyperextract/templates/) 获取所有模板。----## 📚 文档与资源+## 📚 文档- [📖 完整文档](docs/)- [💻 示例代码](examples/)- [🏷️ 模板库](hyperextract/templates/)------## 🤝 贡献与支持--欢迎提交 Issue 和 Pull Request!--如果这个项目对你有帮助,请给我们一个 ⭐!-------*为 AI 社区而构建 ❤️*diff --git a/docs/assets/8-types.jpg b/docs/assets/8-types.jpgnew file mode 100644index 0000000..eaac3b2Binary files /dev/null and b/docs/assets/8-types.jpg differdiff --git a/docs/assets/autotypes-demo.jpeg b/docs/assets/autotypes-demo.jpegdeleted file mode 100644index 6bc3cb1..0000000Binary files a/docs/assets/autotypes-demo.jpeg and /dev/null differdiff --git a/docs/assets/autotypes.png b/docs/assets/autotypes.pngdeleted file mode 100644index 0cf8505..0000000Binary files a/docs/assets/autotypes.png and /dev/null differdiff --git a/docs/assets/before-after-demo.jpeg b/docs/assets/before-after-demo.jpegdeleted file mode 100644index b0f43c7..0000000Binary files a/docs/assets/before-after-demo.jpeg and /dev/null differdiff --git a/docs/assets/concept.png b/docs/assets/concept.pngdeleted file mode 100644index 8745992..0000000Binary files a/docs/assets/concept.png and /dev/null differdiff --git a/docs/assets/domain.png b/docs/assets/domain.pngnew file mode 100644index 0000000..573caccBinary files /dev/null and b/docs/assets/domain.png differdiff --git a/docs/assets/fw.jpeg b/docs/assets/fw.jpegdeleted file mode 100644index 80a844d..0000000Binary files a/docs/assets/fw.jpeg and /dev/null differdiff --git a/docs/assets/fw.png b/docs/assets/fw.pngdeleted file mode 100644index 8408b17..0000000Binary files a/docs/assets/fw.png and /dev/null differdiff --git a/docs/assets/start.jpg b/docs/assets/start.jpgnew file mode 100644index 0000000..c494084Binary files /dev/null and b/docs/assets/start.jpg differdiff --git a/docs/assets/use-cases-demo.jpeg b/docs/assets/use-cases-demo.jpegdeleted file mode 100644index c58a4f2..0000000Binary files a/docs/assets/use-cases-demo.jpeg and /dev/null differdiff --git a/hyperextract-skills/graph-designer/SKILL.md b/hyperextract-skills/graph-designer/SKILL.mdindex 8a382c9..22cd263 100644--- a/hyperextract-skills/graph-designer/SKILL.md+++ b/hyperextract-skills/graph-designer/SKILL.md@@ -87,6 +87,31 @@ display:relation_label: '...'```+## Display Design Principles++**Important**: display.label is used for graph visualization. Edges connect two nodes (source/target), so the edge label should NOT repeat this information.++### Design Principles++| Principle | entity_label | relation_label |+|-----------|--------------|----------------|+| Node | Show core identifier | - |+| Edge | - | Show relation type + optional dimensions |+| Spatio-temporal | - | Show relation type + location + time |++### Recommended relation_label by Type++| Type | relation_label | Example |+|------|----------------|---------|+| graph | `{relation_type}` | `owns` |+| hypergraph | `{event_name}` or `{outcome}` | `Battle of Red Cliffs` |+| spatio_temporal_graph | `{relation_type}@{location}({event_date})` | `meets@Beijing(1985)` |++### Length Reference++- entity_label: Recommended 5-20 characters+- relation_label: Recommended 10-30 characters+## Quick Config Reference### Binary Relations (graph)@@ -134,7 +159,7 @@ identifiers:|------|-----------|| graph | [cases/corporate-ownership.yaml](cases/corporate-ownership.yaml) || hypergraph | [cases/battle-analysis.yaml](cases/battle-analysis.yaml) |-| spatio_temporal_graph | [cases/meeting-records.yaml](cases/meeting-records.yaml) |+| spatio_temporal_graph | [cases/biography-events.yaml](cases/biography-events.yaml) |## Reference Filesdiff --git a/hyperextract-skills/graph-designer/cases/battle-analysis.yaml b/hyperextract-skills/graph-designer/cases/battle-analysis.yamlindex 58fd0dd..d9ae119 100644--- a/hyperextract-skills/graph-designer/cases/battle-analysis.yaml+++ b/hyperextract-skills/graph-designer/cases/battle-analysis.yaml@@ -1,49 +1,116 @@-language: en-name: BattleAnalysis+language: [zh, en]++name: battle_analysistype: hypergraph-tags: [history, military]-description: 'Extract historical battle information'+tags: [military, history, battle, conflict]++description:+ zh: '战役分析模板 - 从军事史料中提取战役的多方参战信息及结果分析'+ en: 'Battle Analysis Template - Extract multi-party battle information and outcome analysis from military historical sources'output:+ description:+ zh: '战役中多方参战力量及其结果分析'+ en: 'Multi-party combat forces and outcome analysis in battles'entities:- description: 'Historical figures and factions'+ description:+ zh: '参战方实体'+ en: 'Combatant entities'fields:- name: nametype: str+ description:+ zh: '参战方名称,如国家名、军队名、指挥官名'+ en: 'Combatant name such as country name, army name, commander name'+ required: true+ - name: category+ type: str+ description:+ zh: '参战方类型:nation(国家)/military(军队)/organization(组织)/commander(指挥官)'+ en: 'Combatant type: nation/military/organization/commander'+ required: true+ - name: description+ type: str+ description:+ zh: '参战方简介,包括实力、背景'+ en: 'Brief description of combatant including strength and background'+ required: falserelations:- description: 'Battle events with participants'+ description:+ zh: '战役事件及其参战方分组'+ en: 'Battle events with grouped combatants'fields:- name: battle_nametype: str+ description:+ zh: '战役名称'+ en: 'Battle name'+ required: true- name: attackerstype: list- description: 'Attacking side'+ description:+ zh: '攻击方参战者列表'+ en: 'List of attackers/attackers'+ required: true- name: defenderstype: list- description: 'Defending side'- - name: commanders- type: list- description: 'Key commanders'- required: false+ description:+ zh: '防御方参战者列表'+ en: 'List of defenders'+ required: true+ - name: action+ type: str+ description:+ zh: '战斗行动描述'+ en: 'Description of battle actions'+ required: true- name: outcometype: str- - name: year+ description:+ zh: '战役结果:victory(胜利)/defeat(失败)/draw(平局)'+ en: 'Battle outcome: victory/defeat/draw'+ required: true+ - name: reasoningtype: str+ description:+ zh: '胜败原因分析'+ en: 'Analysis of victory or defeat reasons'required: falseguideline:- target: 'You are a military history expert. Extract battle information.'+ target:+ zh: '你是一位专业的军事历史分析师,擅长从史料中提取战役的参战方信息和战术分析。'+ en: 'You are a professional military history analyst skilled at extracting combatant information and tactical analysis from historical sources.'rules_for_entities:- - 'Extract all mentioned figures and factions'+ zh:+ - '提取战役中涉及的独立参战方,包括国家、军队、组织或个人'+ - '统一参战方名称:国家用全称,军队用正式番号'+ - '指挥官单独作为实体提取时,应标注其所属阵营'+ - 'description应说明参战方的实力规模和主要特点'+ en:+ - 'Extract independent combatants involved in the battle, including nations, armies, organizations, or individuals'+ - 'Standardize combatant names: use full names for nations, formal designations for armies'+ - 'When extracting commanders as separate entities, indicate their affiliation'+ - 'Description should explain the combatant's strength scale and main characteristics'rules_for_relations:- - 'Identify attackers and defenders clearly'- - 'Include key commanders if mentioned'+ zh:+ - '严格区分攻击方和防御方,避免仅依赖原文的表述顺序'+ - '同一参战方出现在多个列表中(如既是攻击方又是支援方),应选择最核心的角色归类'+ - 'action应描述具体的战斗行动,如:发起进攻、包围、坚守阵地等'+ - 'outcome基于战役最终结果判断,而非某一方的主观声称'+ - 'reasoning应从战略、战术、资源、地形等多角度分析胜败原因'+ en:+ - 'Strictly distinguish attackers and defenders, avoid relying solely on source text order'+ - 'When the same combatant appears in multiple lists, choose the most central role for classification'+ - 'Action should describe specific battle actions such as: launching offensive, encirclement, holding position'+ - 'Outcome should be based on final battle result, not one party's claims'+ - 'Reasoning should analyze victory/defeat from multiple perspectives: strategy, tactics, resources, terrain'identifiers:entity_id: namerelation_id: '{battle_name}'- relation_members: [attackers, defenders, commanders]+ relation_members: [attackers, defenders]display:entity_label: '{name}'- relation_label: '{battle_name}: {outcome}'+ relation_label: '{battle_name}'diff --git a/hyperextract-skills/graph-designer/cases/biography-events.yaml b/hyperextract-skills/graph-designer/cases/biography-events.yamlnew file mode 100644index 0000000..7d32549--- /dev/null+++ b/hyperextract-skills/graph-designer/cases/biography-events.yaml@@ -0,0 +1,132 @@+language: [zh, en]++name: biography_events+type: spatio_temporal_graph+tags: [biography, history, timeline, events]++description:+ zh: '人物传记事件模板 - 从人物传记中提取关键事件的时间线和时空轨迹'+ en: 'Biography Events Template - Extract key events timeline and spatio-temporal trajectory from biographies'++output:+ description:+ zh: '人物的关键事件及其时空轨迹'+ en: "Person's key events and spatio-temporal trajectory"+ entities:+ description:+ zh: '人物及相关实体'+ en: 'Person and related entities'+ fields:+ - name: name+ type: str+ description:+ zh: '人物姓名'+ en: 'Person name'+ required: true+ - name: role+ type: str+ description:+ zh: '主要社会角色/职业身份'+ en: 'Primary social role/occupation'+ required: true+ - name: description+ type: str+ description:+ zh: '人物简介'+ en: 'Brief biography'+ required: false+ relations:+ description:+ zh: '人物的关键事件及其时空信息'+ en: "Person's key events with spatio-temporal information"+ fields:+ - name: source+ type: str+ description:+ zh: '主角姓名'+ en: 'Main character name'+ required: true+ - name: target+ type: str+ description:+ zh: '相关人物/组织/地点'+ en: 'Related person/organization/location'+ required: true+ - name: relation_type+ type: str+ description:+ zh: '事件类型:meets(会面)/collaborates_with(合作)/conflicts_with(冲突)/visits(访问)/inherits(继承)/founded(创立)'+ en: 'Event type: meets/collaborates_with/conflicts_with/visits/inherits/founded'+ required: true+ - name: event_date+ type: str+ description:+ zh: '事件日期,格式如"1985年"或"1985-03-15"'+ en: 'Event date in format like "1985" or "1985-03-15"'+ required: true+ - name: location+ type: str+ description:+ zh: '事件发生地点'+ en: 'Event location'+ required: true+ - name: topic+ type: str+ description:+ zh: '事件主题/内容'+ en: 'Event topic/content'+ required: true+ - name: outcome+ type: str+ description:+ zh: '事件结果或影响'+ en: 'Event result or impact'+ required: false++guideline:+ target:+ zh: '你是一位专业的传记分析师,擅长从人物传记中提取关键事件的时间和空间信息,构建人物的时空轨迹图谱。'+ en: "You are a professional biography analyst skilled at extracting key events' temporal and spatial information from biographies, constructing the person's spatio-temporal trajectory."+ rules_for_entities:+ zh:+ - '提取传主本人作为核心实体'+ - '相关人物包括:家人、朋友、同事、对手、历史人物等'+ - '组织作为实体时使用正式名称'+ - '地点作为实体时使用当时的地名/行政归属'+ - 'role应选择最能代表该人物历史地位的描述'+ en:+ - 'Extract the biographical subject as the core entity'+ - 'Related persons include: family, friends, colleagues, rivals, historical figures'+ - 'Use formal names for organizations as entities'+ - 'Use contemporary place names/administrative divisions for locations as entities'+ - 'Role should be the description that best represents the person's historical significance'+ rules_for_relations:+ zh:+ - '提取具有重大意义的事件,如:人生转折点、重要成就、关键冲突等'+ - '事件日期应精确到可获得的最小粒度:年月日优先,年次之'+ - '对于模糊的时间描述(如"二十世纪中期"),使用推断后的具体年份并标注'+ - '地点应精确到城市级别,除非原文仅提供国家'+ - 'relation_type应根据事件本质选择,而非表面的文字描述'+ - '同一人物对同一目标的多次关系,应按时间分别提取'+ - 'outcome应概括事件的历史意义或对人物后续发展的影响'+ en:+ - 'Extract events of significant meaning such as: turning points, major achievements, key conflicts'+ - 'Event dates should be as precise as available: year-month-day preferred, year minimum'+ - 'For vague time descriptions (e.g., "mid-20th century"), use inferred specific year with annotation'+ - 'Locations should be precise to city level unless source only provides country'+ - 'relation_type should be selected based on event essence, not surface text description'+ - 'Multiple relations between the same person and target should be extracted separately by time'+ - 'Outcome should summarize the event's historical significance or impact on the person'++identifiers:+ entity_id: name+ relation_id: '{source}|{relation_type}|{target}|{event_date}'+ relation_members:+ source: source+ target: target+ time_field: event_date+ location_field: location++display:+ entity_label: '{name}'+ relation_label: '{relation_type}@{location}({event_date})'diff --git a/hyperextract-skills/graph-designer/cases/corporate-ownership.yaml b/hyperextract-skills/graph-designer/cases/corporate-ownership.yamlindex 4c81029..b9924d0 100644--- a/hyperextract-skills/graph-designer/cases/corporate-ownership.yaml+++ b/hyperextract-skills/graph-designer/cases/corporate-ownership.yaml@@ -1,40 +1,104 @@-language: en-name: CorporateOwnershipGraph+language: [zh, en]++name: corporate_ownershiptype: graph-tags: [finance, corporate]-description: 'Extract company ownership and control relationships'+tags: [business, finance, corporate, ownership]++description:+ zh: '公司股权关系模板 - 从企业文档中提取公司间的股权持有和投资关系'+ en: 'Corporate Ownership Template - Extract ownership and investment relationships between companies from corporate documents'output:+ description:+ zh: '公司间的股权和投资关系网络'+ en: 'Corporate ownership and investment relationship network'entities:- description: 'Company and person entities'+ description:+ zh: '组织/公司实体'+ en: 'Organization/company entities'fields:- name: nametype: str- description: 'Entity name'+ description:+ zh: '公司全称,使用官方注册名称'+ en: 'Full company name as officially registered'+ required: true- name: categorytype: str- description: 'Company or Person'+ description:+ zh: '公司类型:母公司(parent_company)/子公司(subsidiary)/关联公司(affiliate)'+ en: 'Company type: parent_company/subsidiary/affiliate'+ required: true+ - name: description+ type: str+ description:+ zh: '公司简介,包括主营业务、成立时间等'+ en: 'Company description including main business and establishment date'+ required: falserelations:- description: 'Ownership and control relationships'+ description:+ zh: '公司间的股权和投资关系'+ en: 'Ownership and investment relationships between companies'fields:- name: sourcetype: str+ description:+ zh: '控股方/投资方公司名称'+ en: 'Name of the controlling/investing company'+ required: true- name: targettype: str+ description:+ zh: '被控股方/被投资方公司名称'+ en: 'Name of the controlled/invested company'+ required: true- name: relation_typetype: str- - name: percentage+ description:+ zh: '关系类型:owns(控股)/has_subsidiary(拥有子公司)/invests_in(投资)/acquired_by(被收购)'+ en: 'Relationship type: owns/has_subsidiary/invests_in/acquired_by'+ required: true+ - name: ownership_percentage+ type: str+ description:+ zh: '持股比例,如"51%"、"25%"'+ en: 'Ownership percentage, e.g., "51%", "25%"'+ required: false+ - name: descriptiontype: str+ description:+ zh: '关系说明,如股权变更背景'+ en: 'Relationship description such as background of ownership change'required: falseguideline:- target: 'You are a corporate analysis expert. Extract company relationships.'+ target:+ zh: '你是一位专业的企业分析师,擅长从企业年报、公告和新闻中提取公司间的股权关系。'+ en: 'You are a professional corporate analyst skilled at extracting ownership relationships from annual reports, announcements, and news.'rules_for_entities:- - 'Extract company and person entities only'- - 'Category must be Company or Person'+ zh:+ - '提取具有独立法人资格的公司实体'+ - '统一公司名称:全称优先,避免使用简称除非原文明确使用简称'+ - '公司类型应根据持股比例和实际控制关系判断,而非仅依赖原文描述'+ - 'description应简洁,聚焦于主营业务和关键特征'+ en:+ - 'Extract corporate entities with independent legal status'+ - 'Use full company names consistently; use abbreviations only when explicitly used in source text'+ - 'Determine company type based on shareholding ratio and actual control relationship, not just source text description'+ - 'Description should be concise, focusing on main business and key characteristics'rules_for_relations:- - 'Only extract ownership and control relationships'- - 'Common types: owns, controls, invested_in, subsidiary_of'+ zh:+ - '仅提取文本中明确声明的股权关系,避免推测'+ - '严格区分直接持股和间接持股,在relation_type中体现'+ - '同一公司对同一目标的多层股权关系应分别提取'+ - '持股比例应与原文保持一致,注意区分直接持股比例和间接持股比例'+ - '关系描述应说明股权取得方式(如:IPO前投资、定向增发、公开市场收购)'+ en:+ - 'Extract only explicitly stated ownership relationships, avoid speculation'+ - 'Strictly distinguish between direct and indirect ownership, reflect in relation_type'+ - 'Multiple layers of ownership between the same companies should be extracted separately'+ - 'Ownership percentage should match source text, note the difference between direct and indirect shareholding'+ - 'Relationship description should explain the acquisition method (e.g., pre-IPO investment, private placement, public market acquisition)'identifiers:entity_id: name@@ -44,5 +108,5 @@ identifiers:target: targetdisplay:- entity_label: '{name} ({category})'+ entity_label: '{name}'relation_label: '{relation_type}'diff --git a/hyperextract-skills/graph-designer/cases/meeting-records.yaml b/hyperextract-skills/graph-designer/cases/meeting-records.yamldeleted file mode 100644index 04f847e..0000000--- a/hyperextract-skills/graph-designer/cases/meeting-records.yaml+++ /dev/null@@ -1,49 +0,0 @@-language: en-name: MeetingRecords-type: spatio_temporal_graph-tags: [business, collaboration]-description: 'Extract meeting information'--output:- entities:- description: 'People and organizations'- fields:- - name: name- type: str- relations:- description: 'Meeting events'- fields:- - name: meeting_title- type: str- - name: attendees- type: list- - name: meeting_date- type: str- - name: location- type: str- required: false- - name: summary- type: str- required: false--guideline:- target: 'You are a business analyst. Extract meeting information.'- rules_for_entities:- - 'Extract all mentioned participants'- rules_for_relations:- - 'Extract time and location if mentioned'- rules_for_time:- - 'Use ISO format when possible'- rules_for_location:- - 'Keep structured locations as-is'--identifiers:- entity_id: name- relation_id: '{meeting_title}|{meeting_date}'- relation_members: attendees- time_field: meeting_date- location_field: location--display:- entity_label: '{name}'- relation_label: '{meeting_title} @ {meeting_date}'diff --git a/hyperextract-skills/record-designer/SKILL.md b/hyperextract-skills/record-designer/SKILL.mdindex 054cb09..f26614a 100644--- a/hyperextract-skills/record-designer/SKILL.md+++ b/hyperextract-skills/record-designer/SKILL.md@@ -2,7 +2,7 @@name: record-designerdescription: |Design YAML extraction templates for record types (model, list, set).- Use when user says: "design model template", "create list extraction", "extract ordered items", "deduplicate entities".+ Use when user says: "design model template", "create item list extraction", "extract items", "deduplicate entities".Trigger: User wants to extract structured records, lists, or deduplicated entities.Skip: User mentions graph, relations, or knowledge graph (use graph-designer instead).---@@ -41,7 +41,7 @@ Receive design specs from brainstorm:| Type | Identifiers | Use Case ||------|-------------|----------|| model | Not needed | Single object |-| list | Not needed | Ordered items |+| list | Not needed | List of items || set | `item_id` required | Deduplicated entities |## Output Templatediff --git a/hyperextract-skills/record-designer/cases/earnings-summary.yaml b/hyperextract-skills/record-designer/cases/earnings-summary.yamlindex ec7d036..4234fc2 100644--- a/hyperextract-skills/record-designer/cases/earnings-summary.yaml+++ b/hyperextract-skills/record-designer/cases/earnings-summary.yaml@@ -1,38 +1,68 @@-language: en-name: EarningsCallSummary+language: [zh, en]++name: earnings_summarytype: model-tags: [finance]-description: 'Extract key financial metrics from earnings call transcripts'+tags: [finance, financial-report, earnings]++description:+ zh: '财报摘要模板 - 从财报文本中提取关键财务指标和业绩摘要'+ en: 'Earnings Summary Template - Extract key financial metrics and performance summary from financial reports'output:- description: 'Earnings call summary'+ description:+ zh: '单个公司的财务业绩摘要'+ en: 'Financial performance summary of a single company'fields:- name: company_nametype: str- description: 'Company name'+ description:+ zh: '公司全称,使用财报中的官方名称'+ en: 'Full company name as stated in the financial report'+ required: true+ - name: reporting_period+ type: str+ description:+ zh: '报告期间,格式如"2024Q3"或"2024年上半年"'+ en: 'Reporting period in format like "2024Q3" or "H1 2024"'required: true- default: ''- name: revenuetype: str- description: 'Revenue amount'- required: false- default: 'N/A'- - name: quarter+ description:+ zh: '营业收入金额,如"100"亿元'+ en: 'Revenue amount, e.g., "100" in hundred millions'+ required: true+ - name: net_profittype: str- description: 'Fiscal quarter'+ description:+ zh: '净利润金额,如"15"亿元'+ en: 'Net profit amount, e.g., "15" in hundred millions'required: true- default: ''- - name: key_highlights- type: list- description: 'Key highlights from the call'+ - name: notes+ type: str+ description:+ zh: '其他重要备注或说明'+ en: 'Other important notes or remarks'required: falseguideline:- target: 'You are a financial analyst. Extract key metrics from earnings calls.'+ target:+ zh: '你是一位专业的财务分析师,负责从财报文本中准确提取核心财务数据。请确保数据的精确性和完整性。'+ en: 'You are a professional financial analyst responsible for accurately extracting core financial data from financial reports. Ensure precision and completeness of the data.'rules:- - 'Extract revenue in the format shown in text (including currency)'- - 'Use N/A if revenue is not mentioned'- - 'Extract quarter as Q1/Q2/Q3/Q4 format'+ zh:+ - '提取精确的营收和净利润数字,确保与财务报表一致'+ - '严格区分本报告期数据与历史/对比期数据,避免混淆'+ - '报告期格式应与原文保持一致,使用标准格式如"2024Q3"'+ - '金额默认单位为亿元人民币,数值保留原始精度'+ - '备注字段用于记录其他重要信息,如:数据来源、特殊事项、异常情况等'+ en:+ - 'Extract precise revenue and net profit figures, ensuring consistency with financial statements'+ - 'Strictly distinguish between current reporting period data and historical/comparative period data to avoid confusion'+ - 'Reporting period format should match the source text, use standard formats like "2024Q3"'+ - 'Amounts are in hundred millions of CNY by default, preserve original precision'+ - 'Use the notes field to record other important information such as data sources, special events, or anomalies'++identifiers: {}display:- label: '{company_name} - {quarter}'+ label: '{company_name}'diff --git a/hyperextract-skills/record-designer/cases/entity-registry.yaml b/hyperextract-skills/record-designer/cases/entity-registry.yamlindex 4ec9870..7a0bb48 100644--- a/hyperextract-skills/record-designer/cases/entity-registry.yaml+++ b/hyperextract-skills/record-designer/cases/entity-registry.yaml@@ -1,32 +1,57 @@-language: en-name: EntityRegistry+language: [zh, en]++name: entity_registrytype: set-tags: [entity, registry]-description: 'Extract and deduplicate entity mentions'+tags: [entity, knowledge, extraction]++description:+ zh: '实体注册表模板 - 从文本中识别并提取关键实体'+ en: 'Entity Registry Template - Identify and extract key entities from text'output:- description: 'Entity registry'+ description:+ zh: '从文本中提取的实体清单'+ en: 'Entity list extracted from text'fields:- name: nametype: str- description: 'Entity name'- - name: category+ description:+ zh: '实体名称,使用最稳定、最清晰的官方称呼'+ en: 'Entity name, using the most stable and clear official designation'+ required: true+ - name: entity_typetype: str- description: 'Entity type'- - name: source+ description:+ zh: '实体类型,如"人物"、"机构"、"地点"、"产品"等'+ en: 'Entity type such as "person", "organization", "location", "product"'+ required: true+ - name: descriptiontype: str- description: 'Source document'+ description:+ zh: '实体简要描述,说明其身份或作用'+ en: 'Brief description of the entity identity or role'required: falseguideline:- target: 'You are a knowledge engineer. Extract entities and deduplicate.'+ target:+ zh: '你是一位专业的实体识别专家,擅长从各类文本中准确识别有意义的实体。'+ en: 'You are a professional entity recognition expert skilled at accurately identifying meaningful entities from various texts.'rules:- - 'Extract all mentioned entities'- - 'Deduplicate by name'- - 'Keep first occurrence of each entity'+ zh:+ - '提取具有独立指代意义的实体,即能够唯一标识某个对象'+ - '统一同一实体的不同表述:全称/简称/别称应合并,name字段使用最正式的名称'+ - '不要将纯代词(他、她、它)、泛称("某公司")、无实际指代的短语作为实体'+ - '实体类型分类要准确且一致'+ - 'description应精炼概括实体身份或作用'+ en:+ - 'Extract entities with independent referential meaning, i.e., those that can uniquely identify an object'+ - 'Unify different expressions of the same entity: full names/abbreviations/aliases should be merged, use the most formal name in the name field'+ - 'Do not treat pure pronouns (he, she, it), generic terms ("a certain company"), or non-referential phrases as entities'+ - 'Entity type classification must be accurate and consistent'+ - 'Description should be a concise summary of the entity identity or role'identifiers:item_id: namedisplay:- label: '{name} ({category})'+ label: '{name} ({entity_type})'diff --git a/hyperextract-skills/record-designer/cases/product-features.yaml b/hyperextract-skills/record-designer/cases/product-features.yamlindex ea9fc38..23b7898 100644--- a/hyperextract-skills/record-designer/cases/product-features.yaml+++ b/hyperextract-skills/record-designer/cases/product-features.yaml@@ -1,28 +1,67 @@-language: en-name: ProductFeatures+language: [zh, en]++name: product_featurestype: list-tags: [product, features]-description: 'Extract product features'+tags: [product, feature, analysis]++description:+ zh: '产品特性列表模板 - 从产品文档、评测或描述中提取结构化的产品特性列表'+ en: 'Product Features List Template - Extract structured product feature lists from product documentation, reviews, or descriptions'output:- description: 'Product feature list'+ description:+ zh: '从文本中提取的产品特性列表'+ en: 'List of product features extracted from the text'fields:- - name: order- type: int- description: 'Feature order'- name: featuretype: str- description: 'Feature name'- - name: description+ description:+ zh: '产品特性名称或描述'+ en: 'Product feature name or description'+ required: true+ - name: category+ type: str+ description:+ zh: '特性所属类别,如"性能"、"功能"、"体验"、"安全"等'+ en: 'Category of the feature such as "performance", "functionality", "experience", "security"'+ required: true+ - name: sentimenttype: str- description: 'Feature description'+ description:+ zh: '特性描述的情感倾向:正面/中性/负面'+ en: 'Sentiment of the feature description: positive/neutral/negative'+ required: false+ default: 'neutral'+ - name: evidence+ type: str+ description:+ zh: '原文引用,说明该特性的来源'+ en: 'Quote from the source text supporting this feature'required: falseguideline:- target: 'You are a product analyst. Extract product features.'+ target:+ zh: '你是一位产品分析专家,擅长从各类文本中识别、归类和提取产品特性。'+ en: 'You are a product analysis expert skilled at identifying, categorizing, and extracting product features from various texts.'rules:- - 'Extract features in order they appear'- - 'Include brief description if available'+ zh:+ - '提取所有明确的、具体的产品特性,不遗漏重要特性'+ - '对特性进行合理分类,使用统一的分类标签'+ - '常见分类包括:性能、功能、体验、安全、价格等'+ - '合并语义高度相近的特性,避免重复'+ - '去除纯主观评价、营销话术、泛泛而谈的描述'+ - '保留具体的功能描述和技术指标'+ - '情感倾向判断基于特性描述本身,而非用户评论或主观感受'+ en:+ - 'Extract all explicit and specific product features without omitting important ones'+ - 'Categorize features reasonably using consistent category labels'+ - 'Common categories include: performance, functionality, experience, security, price, etc.'+ - 'Merge highly similar features to avoid redundancy'+ - 'Remove purely subjective evaluations, marketing language, and vague descriptions'+ - 'Preserve specific functional descriptions and technical specifications'+ - 'Sentiment assessment is based on the feature description itself, not user reviews or subjective feelings'++identifiers: {}display:- label: '{order}. {feature}'+ label: '{feature}'diff --git a/hyperextract/cli/README.md b/hyperextract/cli/README.mdnew file mode 100644index 0000000..733f8a3--- /dev/null+++ b/hyperextract/cli/README.md@@ -0,0 +1,290 @@+# Hyper-Extract CLI++> **"Stop reading. Start understanding."**++A powerful command-line tool for extracting structured knowledge from unstructured text.++---++## ⚡ Quick Start++```bash+# Install+pip install hyper-extract++# First-time setup+he config init++# Extract knowledge from a document+he parse document.md -o my_kb -l zh++# Visualize the knowledge graph+he show my_kb++# Search within your knowledge base+he search my_kb "key insights"+```++---++## ⚙️ Configuration++Before using Hyper-Extract, you need to configure your LLM and Embedder settings.++### Interactive Setup (Recommended)++```bash+he config init+```++This will guide you through setting up:+1. LLM configuration (model, API key, base URL)+2. Embedder configuration (model, API key, base URL)++Note: Provider is fixed to OpenAI by default.++### Manual Configuration++#### Configure LLM++```bash+# Basic configuration+he config llm --api-key YOUR_API_KEY++# Specify model and base URL+he config llm --model gpt-4o --api-key YOUR_API_KEY --base-url https://api.openai.com/v1+```++#### Configure Embedder++```bash+# Basic configuration+he config embedder --api-key YOUR_API_KEY++# Specify model+he config embedder --model text-embedding-3-small --api-key YOUR_API_KEY+```++### Environment Variables++You can also use environment variables as an alternative:++```bash+export OPENAI_API_KEY=your_api_key+export OPENAI_BASE_URL=https://api.openai.com/v1 # Optional+```++Environment variables take precedence over config file settings.++### View Current Configuration++```bash+he config show+```++---++## 📄 Parse Command++Extract knowledge from documents into structured knowledge bases.++### Basic Usage++```bash+# Parse with interactive template selection+he parse document.md -o my_kb -l zh++# Parse with specific template+he parse document.md -o my_kb -t general/knowledge_graph -l zh++# Parse with specific method+he parse document.md -o my_kb -m light_rag+```++### Options++- `<input>` - Input file path, directory, or `-` for stdin+- `-o, --output` - Output knowledge base directory (required)+- `-t, --template` - Template ID (omit for interactive selection)+- `-m, --method` - Method template (e.g., `light_rag`, `hyper_rag`)+- `-l, --lang` - Language (`zh` or `en`, required for knowledge templates)+- `-f, --force` - Force overwrite existing output+- `--no-index` - Skip building search index++### List Available Templates++```bash+he list template+he list template -l zh # Filter by language+he list template -a graph # Filter by type+he list template -q finance # Search by keyword+```++### List Available Methods++```bash+he list method+he list method -q rag # Search by keyword+```++---++## 🔍 Other Commands++### Build Search Index++Required for semantic search and chat functionality.++```bash+he build-index my_kb+he build-index my_kb --force # Rebuild existing index+```++### Search Knowledge Base++Perform semantic search within your knowledge base.++```bash+he search my_kb "What are the key findings?"+he search my_kb "key insights" -n 5 # Return top 5 results+```++### Chat with Knowledge Base++Ask questions about your knowledge base.++```bash+# Single query+he talk my_kb -q "What was discussed in the meeting?"++# Interactive mode+he talk my_kb -i+```++### Visualize Knowledge Graph++View your knowledge base as an interactive graph.++```bash+he show my_kb+```++### View Knowledge Base Info++Display statistics and metadata about your knowledge base.++```bash+he info my_kb+```++### Add Knowledge to Existing KB++Append new knowledge to an existing knowledge base.++```bash+he feed my_kb new_document.md+```++---++## 📝 Examples++### Example 1: Extract Financial Data++```bash+# Configure API keys+he config init++# List finance templates+he list template -l zh | grep finance++# Extract earnings report+he parse earnings_report.md -o finance_kb -t finance/earnings_summary -l zh++# Build index for search+he build-index finance_kb++# Search for insights+he search finance_kb "What was the revenue growth?"+```++### Example 2: Extract Legal Contracts++```bash+# List legal templates+he list template -l zh | grep legal++# Extract contract information+he parse contract.md -o legal_kb -t legal/contract_summary -l zh++# View as knowledge graph+he show legal_kb+```++### Example 3: Use Method Templates++```bash+# Use Light RAG method+he parse document.md -o kb -m light_rag++# Use Hyper RAG method+he parse document.md -o kb -m hyper_rag+```++---++## ❓ FAQ++### Q: How do I choose between template and method?++**A:** Use **templates** when you need domain-specific extraction (finance, legal, medicine, etc.). Use **methods** when you want algorithm-driven extraction (RAG-based approaches).++### Q: Why do I need to build an index?++**A:** The index enables semantic search and chat functionality. Without it, you can still extract and visualize knowledge, but search and talk commands won't work.++### Q: How do I switch between languages?++**A:** Use the `-l` or `--lang` option:+- `-l zh` for Chinese+- `-l en` for English++### Q: Can I use custom API endpoints?++**A:** Yes! Configure with `--base-url` to use API-compatible endpoints:+```bash+he config llm --api-key YOUR_KEY --base-url https://your-custom-api.com/v1+```++### Q: Where is the configuration stored?++**A:** Configuration is stored in `~/.he/config.toml`++### Q: How do I update my API key?++**A:** Simply run the configuration command again:+```bash+he config llm --api-key NEW_API_KEY+```++---++## 🆘 Need Help?++```bash+# View all available commands+he --help++# View help for specific command+he parse --help+he config --help+he search --help+```++---++## 📚 Learn More++- [Full Documentation](../README.md)+- [Template Gallery](../templates/)+- [Examples](../examples/)diff --git a/hyperextract/cli/commands/config.py b/hyperextract/cli/commands/config.pyindex 97f6566..e22a607 100644--- a/hyperextract/cli/commands/config.py+++ b/hyperextract/cli/commands/config.py@@ -3,8 +3,9 @@from typing import Optionalfrom rich.console import Consolefrom rich.table import Table+from rich.text import Text-from ..config import ConfigManager, LLM_PROVIDERS, EMBEDDER_PROVIDERS+from ..config import ConfigManagerimport typerconsole = Console()@@ -12,9 +13,84 @@ console = Console()app = typer.Typer(name="config",help="Manage LLM and Embedder configuration",+ invoke_without_command=True,)+def config_callback(+ ctx: typer.Context,+):+ """Show configuration help when running 'he config' without subcommand."""+ if ctx.invoked_subcommand is not None:+ return++ from ..utils import LOGO++ console.print()+ console.print(Text(LOGO, style="bold cyan"))++ from rich.rule import Rule++ console.print(Rule(style="cyan dim"))+ console.print()++ title_text = Text("CONFIGURATION", style="bold cyan")+ desc_text = Text("Manage LLM and Embedder settings", style="dim")++ header = Table(box=None, show_header=False, pad_edge=False)+ header.add_column(no_wrap=True)+ header.add_column(style="dim white", no_wrap=True)+ header.add_row(title_text, desc_text)++ console.print(header)+ console.print()+ console.print(Rule(style="cyan dim"))+ console.print()++ console.print("[bold cyan]Available Commands:[/bold cyan]")+ console.print()++ commands_info = [+ ("he config init", "Interactive configuration setup (recommended for first-time users)"),+ ("he config show", "Display current configuration"),+ ("he config llm", "Configure LLM settings"),+ ("he config embedder", "Configure Embedder settings"),+ ]++ for cmd, desc in commands_info:+ console.print(f" [green]{cmd:<30}[/green] {desc}")++ console.print()+ console.print(Rule(style="cyan dim"))+ console.print()++ console.print("[bold cyan]Quick Start:[/bold cyan]")+ console.print()+ console.print(" [yellow]1.[/yellow] Run [green]he config init[/green] for interactive setup")+ console.print(" [yellow]2.[/yellow] Or configure individually:")+ console.print(" [green]he config llm --api-key YOUR_KEY[/green]")+ console.print(" [green]he config embedder --api-key YOUR_KEY[/green]")+ console.print()++ console.print("[bold cyan]Environment Variables (alternative):[/bold cyan]")+ console.print()+ console.print(" [green]OPENAI_API_KEY[/green] - OpenAI API key (used if not set in config)")+ console.print(" [green]OPENAI_BASE_URL[/green] - Custom API base URL (optional)")+ console.print()++ console.print(Rule(style="cyan dim"))+ console.print()++ hint_text = Text("💡 Tip: Run ", style="dim")+ hint_text.append("he config <command> --help", style="bold cyan")+ hint_text.append(" for detailed command usage", style="dim")+ console.print(hint_text)+ console.print()++ raise typer.Exit()++def _show_config():"""Show current configuration."""config = ConfigManager()@@ -22,24 +98,24 @@ def _show_config():table = Table(title="Hyper-Extract Configuration")table.add_column("Service", style="cyan", width=15)- table.add_column("Provider", style="green", width=15)- table.add_column("Model", style="yellow", width=25)- table.add_column("API Key", style="magenta", width=20)+ table.add_column("Model", style="yellow", width=30)+ table.add_column("API Key", style="magenta", width=25)+ table.add_column("Base URL", style="green", width=25)llm_cfg = cfg["llm"]emb_cfg = cfg["embedder"]table.add_row("LLM",- llm_cfg["provider"],llm_cfg["model"],llm_cfg["api_key"][:10] + "..." if llm_cfg["api_key"] else "(not set)",+ llm_cfg["base_url"] or "(default)",)table.add_row("Embedder",- emb_cfg["provider"],emb_cfg["model"],emb_cfg["api_key"][:10] + "..." if emb_cfg["api_key"] else "(not set)",+ emb_cfg["base_url"] or "(default)",)console.print(table)@@ -55,12 +131,6 @@ def show(@app.command(name="llm")def llm(- provider: str = typer.Option(- None,- "--provider",- "-p",- help=f"LLM provider ({'/'.join(LLM_PROVIDERS)})",- ),api_key: Optional[str] = typer.Option(None,"--api-key",@@ -90,7 +160,6 @@ def llm(table = Table(title="LLM Configuration", show_header=False)table.add_column("Key", style="cyan")table.add_column("Value", style="green")- table.add_row("Provider", cfg.provider)table.add_row("Model", cfg.model)table.add_row("API Key", cfg.api_key[:10] + "..." if cfg.api_key else "(not set)")table.add_row("Base URL", cfg.base_url or "(default)")@@ -103,7 +172,6 @@ def llm(returnconfig.set_llm(- provider=provider,model=model,api_key=api_key,base_url=base_url,@@ -113,12 +181,6 @@ def llm(@app.command(name="embedder")def embedder(- provider: str = typer.Option(- None,- "--provider",- "-p",- help=f"Embedder provider ({'/'.join(EMBEDDER_PROVIDERS)})",- ),api_key: Optional[str] = typer.Option(None,"--api-key",@@ -148,7 +210,6 @@ def embedder(table = Table(title="Embedder Configuration", show_header=False)table.add_column("Key", style="cyan")table.add_column("Value", style="green")- table.add_row("Provider", cfg.provider)table.add_row("Model", cfg.model)table.add_row("API Key", cfg.api_key[:10] + "..." if cfg.api_key else "(not set)")table.add_row("Base URL", cfg.base_url or "(default)")@@ -161,7 +222,6 @@ def embedder(returnconfig.set_embedder(- provider=provider,api_key=api_key,model=model,base_url=base_url,@@ -180,31 +240,39 @@ def init(console.print()console.print("[bold]Step 1: LLM Configuration[/bold]")- provider = console.input(" Provider [openai]: ") or "openai"- model = console.input(" Model [gpt-4o-mini]: ") or "gpt-4o-mini"- api_key = console.input(" API Key: ", password=True)- base_url = console.input(" Base URL (optional): ")+ model = console.input(" Model (default: gpt-4o-mini): ") or "gpt-4o-mini"++ api_key = None+ while not api_key:+ api_key = console.input(" API Key: ")+ if not api_key:+ console.print(" [red]API Key is required. Please enter your API key.[/red]")++ base_url = console.input(" Base URL (optional, press Enter to skip): ") or Noneconfig.set_llm(- provider=provider,model=model,api_key=api_key,- base_url=base_url or None,+ base_url=base_url,)console.print()console.print("[bold]Step 2: Embedder Configuration[/bold]")- emb_provider = console.input(" Provider [openai]: ") or "openai"- emb_model = console.input(" Model [text-embedding-3-small]: ") or "text-embedding-3-small"- emb_api_key = console.input(" API Key: ", password=True)- emb_base_url = console.input(" Base URL (optional): ")+ emb_model = console.input(" Model (default: text-embedding-3-small): ") or "text-embedding-3-small"++ emb_api_key = None+ while not emb_api_key:+ emb_api_key = console.input(" API Key: ")+ if not emb_api_key:+ console.print(" [red]API Key is required. Please enter your API key.[/red]")++ emb_base_url = console.input(" Base URL (optional, press Enter to skip): ") or Noneconfig.set_embedder(- provider=emb_provider,model=emb_model,api_key=emb_api_key,- base_url=emb_base_url or None,+ base_url=emb_base_url,)console.print()diff --git a/hyperextract/cli/commands/list.py b/hyperextract/cli/commands/list.pyindex fec888b..e1a729c 100644--- a/hyperextract/cli/commands/list.py+++ b/hyperextract/cli/commands/list.py@@ -15,6 +15,14 @@ app = typer.Typer()+def _get_description(cfg) -> str:+ """Get description from template config, handling both str and dict types."""+ desc = cfg.description+ if isinstance(desc, dict):+ return desc.get("zh", desc.get("en", ""))+ return desc or ""++@app.command(name="template")def template(query: Optional[str] = typer.Option(None, "--query", "-q", help="Query to search templates"),@@ -22,7 +30,7 @@ def template(language: Optional[str] = typer.Option(None, "--lang", "-l", help="Filter by language"),include_methods: bool = typer.Option(True, "--include-methods/--no-methods", help="Include method templates"),):- """List all available templates."""+ """List all available templates with detailed information."""results = Gallery.list(filter_by_query=query,filter_by_type=autotype,@@ -37,36 +45,52 @@ def template(languages = [languages]for lang in languages:- templates.append((path, full_name, lang))+ templates.append((path, full_name, lang, cfg.type, _get_description(cfg)))if include_methods:from hyperextract.methods import list_method_cfgsmethod_templates = list_method_cfgs()for name, cfg in method_templates.items():- templates.append((name, cfg.name, "en"))+ templates.append((name, cfg.name, "en", cfg.type, cfg.description))if not templates:console.print("[yellow]No templates found.[/yellow]")return- table = Table(title="Available Templates", show_header=True, header_style="bold magenta")- table.add_column("Template ID", style="cyan", width=35)- table.add_column("Name", style="green", width=30)- table.add_column("Language", style="yellow", width=10)+ table = Table(+ title="Available Templates",+ show_header=True,+ header_style="bold magenta",+ expand=True,+ )+ table.add_column("Template ID", style="cyan", no_wrap=True, width=30)+ table.add_column("Name", style="green", width=20)+ table.add_column("Type", style="yellow", no_wrap=True, width=12)+ table.add_column("Language", style="magenta", no_wrap=True, width=8)+ table.add_column("Description", style="white")- for tid, name, lang in templates:- table.add_row(tid, name, lang)+ for tid, name, lang, template_type, desc in templates:+ desc_text = desc[:60] + "..." if len(desc) > 60 else desc+ table.add_row(tid, name, template_type, lang, desc_text)console.print(table)console.print(f"\n[dim]Total: {len(templates)} templates[/dim]")- console.print("\n[dim]Tip: Use -t/--template to specify a template, or -m/--method for methods.[/dim]")+ console.print()+ console.print("[dim]Type explanations:[/dim]")+ console.print(" [cyan]graph[/cyan] - Entity-relationship graph (binary relations)")+ console.print(" [cyan]hypergraph[/cyan] - Hypergraph (n-ary relations)")+ console.print(" [cyan]list[/cyan] - Key-value list")+ console.print(" [cyan]model[/cyan] - Structured data model")+ console.print(" [cyan]set[/cyan] - Entity collection")+ console.print()+ console.print("[dim]Tip: Use [bold]he parse <input> -t <template_id> -l <lang>[/bold] to extract with a template[/dim]")@app.command(name="method")def method(query: Optional[str] = typer.Option(None, "--query", "-q", help="Query to search methods"),):- """List all available extraction methods."""+ """List all available extraction methods with detailed information."""from hyperextract.methods import list_methodsitems = list_methods()@@ -82,14 +106,27 @@ def method(console.print("[yellow]No methods found.[/yellow]")return- table = Table(title="Available Methods", show_header=True, header_style="bold magenta")- table.add_column("Method", style="cyan", width=20)- table.add_column("Type", style="yellow", width=15)+ table = Table(+ title="Available Methods",+ show_header=True,+ header_style="bold magenta",+ expand=True,+ )+ table.add_column("Method", style="cyan", no_wrap=True, width=18)+ table.add_column("Type", style="yellow", no_wrap=True, width=12)table.add_column("Description", style="green")for name, info in items.items():- table.add_row(f"method/{name}", info["type"], info["description"])+ method_id = f"method/{name}"+ method_type = info["type"]+ description = info["description"]+ table.add_row(method_id, method_type, description)console.print(table)console.print(f"\n[dim]Total: {len(items)} methods[/dim]")- console.print("\n[dim]Tip: Use -m/--method to specify a method template.[/dim]")+ console.print()+ console.print("[dim]Method explanations:[/dim]")+ console.print(" [cyan]graph[/cyan] - Binary relation graph (entity-relationship)")+ console.print(" [cyan]hypergraph[/cyan] - N-ary hypergraph (multi-entity relations)")+ console.print()+ console.print("[dim]Tip: Use [bold]he parse <input> -m <method_name>[/bold] to extract with a method[/dim]")diff --git a/hyperextract/cli/config.py b/hyperextract/cli/config.pyindex f2f7d52..819c1a9 100644--- a/hyperextract/cli/config.py+++ b/hyperextract/cli/config.py@@ -13,20 +13,14 @@ DEFAULT_CONFIG_DIR = Path.home() / ".he"DEFAULT_CONFIG_FILE = DEFAULT_CONFIG_DIR / "config.toml"-LLM_PROVIDERS = ["openai", "anthropic", "google"]-EMBEDDER_PROVIDERS = ["openai", "anthropic", "google"]--@dataclassclass LLMConfig:- provider: str = "openai"model: str = "gpt-4o-mini"api_key: str = ""base_url: str = ""def to_dict(self) -> Dict[str, Any]:return {- "provider": self.provider,"model": self.model,"api_key": self.api_key,"base_url": self.base_url,@@ -35,7 +29,6 @@ class LLMConfig:@classmethoddef from_dict(cls, data: Dict[str, Any]) -> "LLMConfig":return cls(- provider=data.get("provider", "openai"),model=data.get("model", "gpt-4o-mini"),api_key=data.get("api_key", ""),base_url=data.get("base_url", ""),@@ -44,14 +37,12 @@ class LLMConfig:@dataclassclass EmbedderConfig:- provider: str = "openai"model: str = "text-embedding-3-small"api_key: str = ""base_url: str = ""def to_dict(self) -> Dict[str, Any]:return {- "provider": self.provider,"model": self.model,"api_key": self.api_key,"base_url": self.base_url,@@ -60,7 +51,6 @@ class EmbedderConfig:@classmethoddef from_dict(cls, data: Dict[str, Any]) -> "EmbedderConfig":return cls(- provider=data.get("provider", "openai"),model=data.get("model", "text-embedding-3-small"),api_key=data.get("api_key", ""),base_url=data.get("base_url", ""),@@ -104,7 +94,6 @@ class ConfigManager:def get_llm_config(self) -> LLMConfig:"""Get LLM config with environment variable fallback."""config = LLMConfig(- provider=self.llm.provider,model=self.llm.model,api_key=self.llm.api_key or os.environ.get("OPENAI_API_KEY", ""),base_url=self.llm.base_url or os.environ.get("OPENAI_BASE_URL", ""),@@ -114,7 +103,6 @@ class ConfigManager:def get_embedder_config(self) -> EmbedderConfig:"""Get Embedder config with environment variable fallback."""config = EmbedderConfig(- provider=self.embedder.provider,model=self.embedder.model,api_key=self.embedder.api_key or os.environ.get("OPENAI_API_KEY", ""),base_url=self.embedder.base_url or os.environ.get("OPENAI_BASE_URL", ""),@@ -123,14 +111,11 @@ class ConfigManager:def set_llm(self,- provider: Optional[str] = None,model: Optional[str] = None,api_key: Optional[str] = None,base_url: Optional[str] = None,) -> None:"""Set LLM configuration."""- if provider:- self.llm.provider = providerif model:self.llm.model = modelif api_key is not None:@@ -141,14 +126,11 @@ class ConfigManager:def set_embedder(self,- provider: Optional[str] = None,model: Optional[str] = None,api_key: Optional[str] = None,base_url: Optional[str] = None,) -> None:"""Set Embedder configuration."""- if provider:- self.embedder.provider = providerif model:self.embedder.model = modelif api_key is not None:diff --git a/hyperextract/utils/client.py b/hyperextract/utils/client.pyindex 0599a2d..efe0cc9 100644--- a/hyperextract/utils/client.py+++ b/hyperextract/utils/client.py@@ -1,8 +1,8 @@-"""Client Factory - Create LLM and Embedder clients from config file."""+"""Client Factory - Create OpenAI LLM and Embedder clients from config."""import osfrom pathlib import Path-from typing import Optional, Tuple, Union+from typing import Tuplefrom langchain_core.embeddings import Embeddingsfrom langchain_core.language_models.chat_models import BaseChatModel@@ -11,8 +11,8 @@ DEFAULT_CONFIG_DIR = Path.home() / ".he"DEFAULT_CONFIG_FILE = DEFAULT_CONFIG_DIR / "config.toml"-def get_client(config_path: Optional[Union[str, Path]] = None) -> Tuple:- """Get LLM client and Embedder from config file.+def get_client(config_path: str|Path = None) -> Tuple[BaseChatModel, Embeddings]:+ """Get OpenAI LLM client and Embedder from config.Args:config_path: Config file path, default ~/.he/config.toml@@ -34,96 +34,19 @@ def get_client(config_path: Optional[Union[str, Path]] = None) -> Tuple:llm_config = manager.get_llm_config()emb_config = manager.get_embedder_config()- llm_client = _create_llm_client(llm_config)- embedder = _create_embedder(emb_config)+ from langchain_openai import ChatOpenAI, OpenAIEmbeddings- return llm_client, embedder---def _create_llm_client(config) -> "BaseChatModel":- """Create LLM client from config."""- if config.provider == "openai":- from langchain_openai import ChatOpenAI-- return ChatOpenAI(- model=config.model,- api_key=config.api_key or os.environ.get("OPENAI_API_KEY", ""),- base_url=config.base_url or None,- temperature=0,- )- elif config.provider == "anthropic":- try:- from langchain_anthropic import ChatAnthropic- except ImportError:- raise ImportError(- "Anthropic support requires the 'langchain-anthropic' package.\n"- "Install it with one of:\n"- " pip install hyperextract[anthropic]\n"- " pip install langchain-anthropic"- )-- return ChatAnthropic(- model=config.model,- api_key=config.api_key or os.environ.get("ANTHROPIC_API_KEY", ""),- )- elif config.provider == "google":- try:- from langchain_google_genai import ChatGoogleGenerativeAI- except ImportError:- raise ImportError(- "Google support requires the 'langchain-google-genai' package.\n"- "Install it with one of:\n"- " pip install hyperextract[google]\n"- " pip install langchain-google-genai"- )+ llm_client = ChatOpenAI(+ model=llm_config.model,+ api_key=llm_config.api_key or os.environ.get("OPENAI_API_KEY", ""),+ base_url=llm_config.base_url or None,+ temperature=0,+ )- return ChatGoogleGenerativeAI(- model=config.model,- google_api_key=config.api_key or os.environ.get("GOOGLE_API_KEY", ""),- )- else:- raise ValueError(f"Unsupported LLM provider: {config.provider}")+ embedder = OpenAIEmbeddings(+ model=emb_config.model,+ api_key=emb_config.api_key or os.environ.get("OPENAI_API_KEY", ""),+ base_url=emb_config.base_url or None,+ )--def _create_embedder(config) -> "Embeddings":- """Create Embedder from config."""- if config.provider == "openai":- from langchain_openai import OpenAIEmbeddings-- return OpenAIEmbeddings(- model=config.model,- api_key=config.api_key or os.environ.get("OPENAI_API_KEY", ""),- base_url=config.base_url or None,- )- elif config.provider == "anthropic":- try:- from langchain_anthropic import AnthropicEmbeddings- except ImportError:- raise ImportError(- "Anthropic support requires the 'langchain-anthropic' package.\n"- "Install it with one of:\n"- " pip install hyperextract[anthropic]\n"- " pip install langchain-anthropic"- )-- return AnthropicEmbeddings(- model=config.model,- api_key=config.api_key or os.environ.get("ANTHROPIC_API_KEY", ""),- )- elif config.provider == "google":- try:- from langchain_google_genai import GoogleGenerativeAIEmbeddings- except ImportError:- raise ImportError(- "Google support requires the 'langchain-google-genai' package.\n"- "Install it with one of:\n"- " pip install hyperextract[google]\n"- " pip install langchain-google-genai"- )-- return GoogleGenerativeAIEmbeddings(- model=config.model,- google_api_key=config.api_key or os.environ.get("GOOGLE_API_KEY", ""),- )- else:- raise ValueError(f"Unsupported Embedder provider: {config.provider}")+ return llm_client, embedder