Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
docs: dedicated News tab in docs; slim down READMEs
yifanfeng97df8244a2bb2d
diff --git a/README.md b/README.mdindex aca9426..2bc2894 100644--- a/README.md+++ b/README.md@@ -69,33 +69,35 @@ uv tool install hyperextract**2. Configure your provider** (pick one):-**OpenAI:**```bash+# OpenAI (LLM + embeddings in one key)he config init -p openai -k YOUR_OPENAI_API_KEY-```-**Anthropic (Claude):**-```bash-he config llm -p anthropic -k YOUR_ANTHROPIC_API_KEY+# DeepSeek (LLM only — pair with an OpenAI embedder, most cost-effective)+he config llm -p deepseek -k YOUR_DEEPSEEK_API_KEYhe config embedder -p openai -k YOUR_OPENAI_API_KEY++# Local vLLM (free, on-premise)+he config llm -p vllm -u http://localhost:8000/v1 -k dummy -m Qwen/Qwen3.5-9B+he config embedder -p vllm -u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3```-**DeepSeek:**+<details>+<summary><b>More providers</b> — Anthropic (Claude), Alibaba Bailian, OrcaRouter…</summary>+<br>+```bash-he config llm -p deepseek -k YOUR_DEEPSEEK_API_KEY+# Anthropic (Claude) — LLM only+he config llm -p anthropic -k YOUR_ANTHROPIC_API_KEYhe config embedder -p openai -k YOUR_OPENAI_API_KEY-```-**Bailian (Alibaba Cloud):**-```bash+# Alibaba Bailian (Qwen, LLM + embeddings in one key)he config init -p bailian -k YOUR_BAILIAN_API_KEY```-**Local vLLM:**-```bash-he config llm -p vllm -u http://localhost:8000/v1 -k dummy -m Qwen/Qwen3.5-9B-he config embedder -p vllm -u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3-```+> OpenAI and Bailian provide both LLM and embedding models in one API. Anthropic and DeepSeek are LLM-only (pair them with an OpenAI-compatible embedder). DeepSeek is the most cost-effective option (~$0.001-0.005/page).++</details>**3. Extract, query & visualize:**@@ -123,7 +125,7 @@ he search ./output/ "inventions" --tag biographyhe info ./output/ --sources```-> **Which provider should I use?** OpenAI and Bailian provide both LLM and embedding models in one API. Anthropic and DeepSeek are LLM-only (pair them with an OpenAI embedder for search/chat). Local vLLM is free but requires a GPU. DeepSeek is the most cost-effective option (~$0.001-0.005/page vs ~$0.01-0.05/page for OpenAI gpt-4o-mini).+> **Which provider should I use?** OpenAI and Bailian provide both LLM and embedding models in one API; Anthropic and DeepSeek are LLM-only (pair with an OpenAI embedder); local vLLM is free but needs a GPU. Full guide: [Provider System](https://yifanfeng97.github.io/Hyper-Extract/latest/concepts/provider-system/).<details><summary><b>🐍 Python API</b> (click to expand)</summary>@@ -247,18 +249,25 @@ Hyper-Extract uses LangChain structured output with **function calling**. The mo**Embedding models** (semantic search) work with any OpenAI-compatible endpoint: `text-embedding-3-small`, `text-embedding-v4` (Bailian), `bge-m3` (local vLLM).-> **DeepSeek note:** DeepSeek V4 models default to "thinking" mode, which Hyper-Extract auto-disables so structured extraction works. Set `DEEPSEEK_API_KEY`. DeepSeek has no embeddings API — pair it with an OpenAI-compatible embedder:+<details>+<summary><b>Provider notes</b> — DeepSeek & Anthropic pairing</summary>+<br>++> **DeepSeek:** V4 models default to "thinking" mode, which Hyper-Extract auto-disables so structured extraction works. Set `DEEPSEEK_API_KEY`. DeepSeek has no embeddings API:+>> ```python> from hyperextract import create_client> llm, emb = create_client(llm="deepseek", embedder="openai:text-embedding-3-small")> ```-> **Anthropic note:** Claude is used for the **LLM** (set `ANTHROPIC_API_KEY`). Anthropic has no embeddings API, so pair it with an OpenAI-compatible embedder:+> **Anthropic:** Claude is used for the **LLM** (set `ANTHROPIC_API_KEY`, extra: `pip install 'hyperextract[anthropic]'`). No embeddings API:+>> ```python> from hyperextract import create_client> llm, emb = create_client(llm="anthropic", embedder="openai:text-embedding-3-small")> ```-> Requires the extra: `pip install 'hyperextract[anthropic]'`.++</details>> 📖 Full guide: [Provider System & Local Model Support](https://yifanfeng97.github.io/Hyper-Extract/latest/concepts/provider-system/)@@ -333,49 +342,9 @@ identifiers:## 📰 What's New-<!-- News snippets are derived from the latest merged PRs. Update as new releases land. -->--### v0.9.0--- **📄 Rich Document Ingestion** — `he parse` / `he feed` now accept PDF, Word, PowerPoint, Excel, HTML, CSV/JSON/XML, EPUB and more via the optional ingest extra (`pip install "hyperextract[ingest]"`, powered by [MarkItDown](https://github.com/microsoft/markitdown)). Non-UTF-8 text (GBK, etc.) is auto-detected; text-less (scanned) PDFs fail with a clear OCR hint instead of silently ingesting garbage.-- **🧱 `chunk_rag` Baseline Method** — a new zero-extraction method: documents are chunked and embedded as-is, and search returns raw text chunks. Zero LLM cost at ingestion, with full provenance (tags, scoped search, per-document rollback). The chunk-retrieval baseline for corpus Q&A and method benchmarking.-- **🐛 Fixes** — `he tag` crashed on every knowledge abstract (`tag_source` was never exposed on any type); `he search`/`he talk`/`he feed`/`he remove --document` crashed on method-built KAs (`method/*` templates were not resolvable from KA metadata).--### v0.8.1 / v0.8.2--- **🏷️ Source Tags & Scoped Search** — `he tag ./ka/ --source doc-1 --add legal`, then `he search ./ka/ "query" --tag legal` to retrieve only within tagged documents. Works for graph, hypergraph, and set KAs. *(#89, #84)*-- **🛡️ Input Validation** — `he parse` / `he feed` now reject unsupported file types (PDF/Office) with a conversion hint instead of silently ingesting garbage. *(#88)*-- **📦 Document Archive Fix** — re-feeding the same source from a differently-named file no longer accumulates stale copies. *(#89)*--### v0.8.0--- **🔄 Document Upsert** — re-feed an attributed document and its previous version is rolled back automatically: removed facts disappear, shared keys re-merge from surviving sources. *(#84)*-- **📁 Per-File Source Attribution** — `he parse ./docs/` attributes each file by its name automatically; roll back or audit any single file later. An explicit `--source` still overrides.-- **⏱️ Spatiotemporal Provenance** — temporal/spatial/spatio-temporal graphs fully support source attribution and rollback, with deterministic (MERGE_FIELD) replay tests.--<details>-<summary><b>v0.5.0 – v0.7.0</b> — provenance, deletion, incremental index, template validator, GraphML/CSV export</summary>--- **🗑️ Two-Tier Knowledge Deletion** — hard-delete by key (`he remove --node/--edge`, orphan edges pruned) or remove a single wrong fact via LLM-assisted editing (`he remove --edit-node --fact`), with dry-run, key-invariance checks, and automatic backups. *(#84)*-- **📜 Source Attribution & Provenance** — `he feed --source` / `he parse --source` record each document's raw contributions; `he remove --document` rolls back exactly what one document contributed; `he info --sources` shows the ledger. *(#84)*-- **📈 Incremental Everything** — feed/parse/removal/edit patch the vector index in place (only affected vectors re-embedded); `he feed` skips documents whose content hash is unchanged (`--refeed` to force). *(#84)*-- **🧪 `he template validate`** — catch semantic template errors before paying for LLM calls: 9 diagnostic rules, `--json` for CI, `--all` for directories. *(#77)*-- **📊 GraphML & CSV Export** — desktop graph tools and spreadsheets; hypergraphs get a hyperedges table. *(#85)*-- **🌐 OrcaRouter Provider** — one key for 150+ models via `create_client("orcarouter")`. *(#71)*-- **🔐 Config File Permissions** — `~/.he/config.toml` saved `0600`. *(#86)*-- **🔗 Obsidian wikilink fix** — aliases no longer break on `[ ] | # ^`. *(#87)*-- **🛡️ Chunk-Level Fault Isolation** — one failed chunk no longer discards the rest of a multi-chunk extraction. *(#78)*-- **⚡ MCP Python SDK 2.x** — `he-mcp` works on mcp 1.x and 2.x. *(#72, #82)*-- **🔀 Directed-Edge Fix** — `(source, target)` order preserved; custom endpoint field names. *(#74)*-- **🔑 DeepSeek API Key Fix** — `DEEPSEEK_API_KEY` honored on the OpenAI-compatible path. *(#76)*-- **🎓 Education Templates** — `course_concept_graph` + `curriculum_structure`. *(#80)*-- **🧭 Smaller Fixes** — Graph_RAG.search 3-tuple; `he talk -i --top-k`; onboarding/docs overhaul. *(#70, #73, #57)*--</details>--### Archived+**v0.9.0** — 📄 Rich document ingestion: `he parse` / `he feed` now take PDF, Word, PowerPoint, Excel, HTML, EPUB and more (`pip install "hyperextract[ingest]"`) · 🧱 New `chunk_rag` method: zero-extraction chunk baseline with full provenance · 🐛 `he tag` and method-KA command fixes.-See the full changelog in the [GitHub releases](https://github.com/yifanfeng97/hyper-extract/releases).+📰 **[Full release notes](https://yifanfeng97.github.io/Hyper-Extract/latest/news/)** · [All releases](https://github.com/yifanfeng97/hyper-extract/releases)## 📚 Documentation & Resources@@ -384,6 +353,7 @@ See the full changelog in the [GitHub releases](https://github.com/yifanfeng97/h| Full Documentation | [yifanfeng97.github.io/Hyper-Extract](https://yifanfeng97.github.io/Hyper-Extract/latest/) || CLI Guide | [Command-line interface](https://yifanfeng97.github.io/Hyper-Extract/latest/cli/) || Provider System | [Model compatibility & local deployment](https://yifanfeng97.github.io/Hyper-Extract/latest/concepts/provider-system/) |+| News | [Release notes & highlights](https://yifanfeng97.github.io/Hyper-Extract/latest/news/) || Template Gallery | [80+ presets](./hyperextract/templates/presets/) || Examples | [Working code](./examples/) |diff --git a/README_ZH.md b/README_ZH.mdindex b36a25d..973491a 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -69,33 +69,35 @@ uv tool install hyperextract**2. 配置你的 provider**(任选其一):-**OpenAI:**```bash+# OpenAI(一个 key 同时提供 LLM 和 embedding)he config init -p openai -k YOUR_OPENAI_API_KEY-```-**Anthropic (Claude):**-```bash-he config llm -p anthropic -k YOUR_ANTHROPIC_API_KEY+# DeepSeek(仅 LLM——搭配 OpenAI embedder,最经济)+he config llm -p deepseek -k YOUR_DEEPSEEK_API_KEYhe config embedder -p openai -k YOUR_OPENAI_API_KEY++# 本地 vLLM(免费、数据不出本机)+he config llm -p vllm -u http://localhost:8000/v1 -k dummy -m Qwen/Qwen3.5-9B+he config embedder -p vllm -u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3```-**DeepSeek:**+<details>+<summary><b>更多 provider</b> — Anthropic (Claude)、阿里云百炼、OrcaRouter…</summary>+<br>+```bash-he config llm -p deepseek -k YOUR_DEEPSEEK_API_KEY+# Anthropic (Claude) —— 仅 LLM+he config llm -p anthropic -k YOUR_ANTHROPIC_API_KEYhe config embedder -p openai -k YOUR_OPENAI_API_KEY-```-**百炼(阿里云):**-```bash+# 百炼(阿里云,Qwen,一个 key 同时提供 LLM 和 embedding)he config init -p bailian -k YOUR_BAILIAN_API_KEY```-**本地 vLLM:**-```bash-he config llm -p vllm -u http://localhost:8000/v1 -k dummy -m Qwen/Qwen3.5-9B-he config embedder -p vllm -u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3-```+> OpenAI 和百炼同时提供 LLM 和 embedding 模型;Anthropic 和 DeepSeek 仅提供 LLM(需搭配 OpenAI 兼容 embedder)。DeepSeek 最经济(约 $0.001-0.005/页)。++</details>**3. 提取、查询与可视化:**@@ -123,7 +125,7 @@ he search ./output/ "作品" --tag biographyhe info ./output/ --sources```-> **该用哪个 provider?** OpenAI 和百炼同时提供 LLM 和 embedding 模型。Anthropic 和 DeepSeek 仅提供 LLM(搜索/聊天功能需搭配 OpenAI 的 embedder)。本地 vLLM 免费但需要 GPU。DeepSeek 最经济(约 $0.001-0.005/页,而 OpenAI gpt-4o-mini 约 $0.01-0.05/页)。+> **该用哪个 provider?** OpenAI 和百炼同时提供 LLM 和 embedding 模型;Anthropic 和 DeepSeek 仅提供 LLM(需搭配 OpenAI 兼容 embedder);本地 vLLM 免费但需要 GPU。完整指南:[Provider 系统](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/concepts/provider-system/)。<details><summary><b>🐍 Python API</b>(点击展开)</summary>@@ -247,18 +249,25 @@ Hyper-Extract 通过 LangChain 结构化输出的 **function calling** 方法工**嵌入模型**(语义搜索)支持任意 OpenAI 兼容端点:`text-embedding-3-small`、`text-embedding-v4`(百炼)、`bge-m3`(本地 vLLM)。-> **DeepSeek 说明:** V4 模型默认开启 "thinking" 模式,Hyper-Extract 会自动关闭以保证结构化抽取可用。设置 `DEEPSEEK_API_KEY`。DeepSeek 没有嵌入接口,请搭配 OpenAI 兼容的嵌入模型使用:+<details>+<summary><b>Provider 说明</b> — DeepSeek 与 Anthropic 的搭配方式</summary>+<br>++> **DeepSeek:** V4 模型默认开启 "thinking" 模式,Hyper-Extract 会自动关闭以保证结构化抽取可用。设置 `DEEPSEEK_API_KEY`。DeepSeek 没有嵌入接口:+>> ```python> from hyperextract import create_client> llm, emb = create_client(llm="deepseek", embedder="openai:text-embedding-3-small")> ```-> **Anthropic 说明:** Claude 仅用于 **LLM**(设置 `ANTHROPIC_API_KEY`)。Anthropic 没有嵌入接口,请搭配 OpenAI 兼容的嵌入模型使用:+> **Anthropic:** Claude 仅用于 **LLM**(设置 `ANTHROPIC_API_KEY`,额外依赖:`pip install 'hyperextract[anthropic]'`)。没有嵌入接口:+>> ```python> from hyperextract import create_client> llm, emb = create_client(llm="anthropic", embedder="openai:text-embedding-3-small")> ```-> 需安装额外依赖:`pip install 'hyperextract[anthropic]'`。++</details>> 📖 完整指南:[Provider 系统与本地模型支持](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/concepts/provider-system/)@@ -290,8 +299,8 @@ Hyper-Extract 通过 LangChain 结构化输出的 **function calling** 方法工Hyper-Extract 采用**三层架构**:-- **Auto-Types** — 8 种强类型数据结构(模型、列表、集合、图谱、超图、时序图、空间图、时空图)-- **Methods** — 提取算法:KG-Gen、GraphRAG、LightRAG、Hyper-RAG、Cog-RAG 等+- **Auto-Types** — 9 种强类型数据结构(模型、列表、集合、图谱、超图、时序图、空间图、时空图、文档语料)+- **Methods** — 提取与检索算法:chunk_rag 基线、KG-Gen、GraphRAG、LightRAG、Hyper-RAG、Cog-RAG 等- **Templates** — 覆盖 6 大领域的 80+ 预设模板,零代码配置<img src="docs/assets/arch.jpg" alt="系统架构" width="750" style="max-width: 100%;">@@ -333,49 +342,9 @@ identifiers:## 📰 最新动态-<!-- 以下摘要来自最近合并的 PR,随版本更新而更新。 -->--### v0.9.0--- **📄 丰富的文档输入** — `he parse` / `he feed` 现在支持 PDF、Word、PowerPoint、Excel、HTML、CSV/JSON/XML、EPUB 等格式,安装可选依赖即可启用(`pip install "hyperextract[ingest]"`,由 [MarkItDown](https://github.com/microsoft/markitdown) 驱动)。非 UTF-8 文本(GBK 等)自动识别编码;无文字层的扫描版 PDF 会给出明确的 OCR 提示,而不是静默摄入乱码。-- **🧱 `chunk_rag` 基线方法** — 全新的零提取方法:文档直接分块并向量化,检索返回原始文本块。摄入零 LLM 成本,且具备完整溯源能力(标签、范围检索、按文档回滚)。语料问答与方法对比的块检索基线。-- **🐛 修复** — `he tag` 在所有知识库上都会崩溃(`tag_source` 此前从未在任何类型上实现);`he search`/`he talk`/`he feed`/`he remove --document` 在方法型知识库上会崩溃(`method/*` 模板无法从元数据解析)。--### v0.8.1 / v0.8.2--- **🏷️ 来源标签与范围检索** — `he tag ./ka/ --source doc-1 --add legal`,然后 `he search ./ka/ "query" --tag legal` 即可只在已标注文档范围内检索。适用于 graph、hypergraph 和 set。*(#89, #84)*-- **🛡️ 输入类型校验** — `he parse` / `he feed` 现在会拒绝不支持的文件类型(PDF/Office)并给出转换提示,而不是静默摄入乱码。*(#88)*-- **📦 文档存档修复** — 同一来源换文件名重喂不再积累陈旧副本。*(#89)*--### v0.8.0--- **🔄 文档级 Upsert** — 重新喂入已标注的文档时自动回滚其旧版本:更新后删掉的事实不再残留,共享 key 从幸存来源重新合并。*(#84)*-- **📁 逐文件来源标注** — `he parse ./docs/` 自动按文件名标注每个文件;之后可单独回滚或审计任意文件。显式 `--source` 仍可覆盖。*(#84)*-- **⏱️ 时空图谱溯源** — 时序/空间/时空图谱完整支持来源标注与回滚,MERGE_FIELD 重放测试保证确定性。*(#84)*--<details>-<summary><b>v0.5.0 – v0.7.0</b> — 溯源、删除、增量索引、模板校验、GraphML/CSV 导出</summary>--- **🗑️ 两级知识删除** — 按 key 硬删除(`he remove --node/--edge`,孤儿边自动清理),或借助大模型移除单条错误事实(`he remove --edit-node --fact`),支持 dry-run、key 不变性校验与自动备份。*(#84)*-- **📜 来源标注与溯源** — `he feed --source` / `he parse --source` 记录每份文档的原始贡献;`he remove --document` 精确回滚单份文档的贡献;`he info --sources` 查看来源账本。*(#84)*-- **📈 全链路增量** — feed/parse/删除/编辑均原地修补向量索引(只重嵌受影响向量);`he feed` 对内容哈希未变的文档自动跳过(`--refeed` 强制重喂)。*(#84)*-- **🧪 `he template validate`** — 在调用大模型前发现模板语义错误:9 条诊断规则、`--json` 接入 CI、`--all` 批量校验。*(#77)*-- **📊 GraphML 与 CSV 导出** — 对接桌面图分析工具与表格软件;超图附带超边表。*(#85)*-- **🌐 OrcaRouter 提供商** — 一个 key 调用 150+ 模型:`create_client("orcarouter")`。*(#71)*-- **🔐 配置文件权限** — `~/.he/config.toml` 以 `0600` 保存。*(#86)*-- **🔗 Obsidian 链接修复** — wikilink 别名不再因 `[ ] | # ^` 断链。*(#87)*-- **🛡️ Chunk 级故障隔离** — 单个 chunk 失败不再丢弃多 chunk 抽取的其余结果。*(#78)*-- **⚡ MCP Python SDK 2.x** — `he-mcp` 同时兼容 mcp 1.x / 2.x。*(#72, #82)*-- **🔀 有向边修复** — 保留 `(source, target)` 方向;支持自定义端点字段名。*(#74)*-- **🔑 DeepSeek API Key 修复** — OpenAI 兼容路径正确读取 `DEEPSEEK_API_KEY`。*(#76)*-- **🎓 教育领域模板** — `course_concept_graph` 与 `curriculum_structure`。*(#80)*-- **🧭 其他修复** — Graph_RAG.search 3-tuple;`he talk -i` 支持 `--top-k`;入门/开发安装/结构化输出文档更新。*(#70, #73, #57)*--</details>--### 更早版本+**v0.9.0** — 📄 丰富的文档输入:`he parse` / `he feed` 现在支持 PDF、Word、PowerPoint、Excel、HTML、EPUB 等格式(`pip install "hyperextract[ingest]"`)· 🧱 新增 `chunk_rag` 方法:零提取成本的块检索基线,具备完整溯源 · 🐛 修复 `he tag` 与方法型知识库的命令崩溃问题。-完整更新日志请参阅 [GitHub releases](https://github.com/yifanfeng97/hyper-extract/releases)。+📰 **[完整版本说明](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/news/)** · [全部 Releases](https://github.com/yifanfeng97/hyper-extract/releases)## 📚 文档与资源@@ -384,6 +353,7 @@ identifiers:| 完整文档 | [yifanfeng97.github.io/Hyper-Extract](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/) || CLI 指南 | [命令行界面](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/cli/) || Provider 系统 | [模型兼容性与本地部署](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/concepts/provider-system/) |+| 新闻动态 | [版本说明与亮点](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/news/) || 模板画廊 | [80+ 预设模板](./hyperextract/templates/presets/) || 示例代码 | [可运行示例](./examples/) |diff --git a/docs/en/news.md b/docs/en/news.mdnew file mode 100644index 0000000..e0527b7--- /dev/null+++ b/docs/en/news.md@@ -0,0 +1,46 @@+# News++Release notes and highlights. For a complete changelog, see the [GitHub releases](https://github.com/yifanfeng97/hyper-extract/releases).++---++## v0.9.0 — Rich Document Ingestion & chunk_rag Baseline++- **📄 Rich Document Ingestion** — `he parse` / `he feed` now accept PDF, Word, PowerPoint, Excel, HTML, CSV/JSON/XML, EPUB and more via the optional ingest extra (`pip install "hyperextract[ingest]"`, powered by [MarkItDown](https://github.com/microsoft/markitdown)). Non-UTF-8 text (GBK, etc.) is auto-detected; text-less (scanned) PDFs fail with a clear OCR hint instead of silently ingesting garbage.+- **🧱 `chunk_rag` Baseline Method** — a new zero-extraction method: documents are chunked and embedded as-is, and search returns raw text chunks. Zero LLM cost at ingestion, with full provenance (tags, scoped search, per-document rollback). The chunk-retrieval baseline for corpus Q&A and method benchmarking.+- **🐛 Fixes** — `he tag` crashed on every knowledge abstract (`tag_source` was never exposed on any type); `he search`/`he talk`/`he feed`/`he remove --document` crashed on method-built KAs (`method/*` templates were not resolvable from KA metadata).++---++## v0.8.1 / v0.8.2++- **🏷️ Source Tags & Scoped Search** — `he tag ./ka/ --source doc-1 --add legal`, then `he search ./ka/ "query" --tag legal` to retrieve only within tagged documents. Works for graph, hypergraph, and set KAs. *(#89, #84)*+- **🛡️ Input Validation** — `he parse` / `he feed` now reject unsupported file types with a conversion hint instead of silently ingesting garbage. *(#88)*+- **📦 Document Archive Fix** — re-feeding the same source from a differently-named file no longer accumulates stale copies. *(#89)*++---++## v0.8.0++- **🔄 Document Upsert** — re-feed an attributed document and its previous version is rolled back automatically: removed facts disappear, shared keys re-merge from surviving sources. *(#84)*+- **📁 Per-File Source Attribution** — `he parse ./docs/` attributes each file by its name automatically; roll back or audit any single file later. An explicit `--source` still overrides.+- **⏱️ Spatiotemporal Provenance** — temporal/spatial/spatio-temporal graphs fully support source attribution and rollback, with deterministic (MERGE_FIELD) replay tests.++---++## v0.5.0 – v0.7.0++- **🗑️ Two-Tier Knowledge Deletion** — hard-delete by key (`he remove --node/--edge`, orphan edges pruned) or remove a single wrong fact via LLM-assisted editing (`he remove --edit-node --fact`), with dry-run, key-invariance checks, and automatic backups. *(#84)*+- **📜 Source Attribution & Provenance** — `he feed --source` / `he parse --source` record each document's raw contributions; `he remove --document` rolls back exactly what one document contributed; `he info --sources` shows the ledger. *(#84)*+- **📈 Incremental Everything** — feed/parse/removal/edit patch the vector index in place (only affected vectors re-embedded); `he feed` skips documents whose content hash is unchanged (`--refeed` to force). *(#84)*+- **🧪 `he template validate`** — catch semantic template errors before paying for LLM calls: 9 diagnostic rules, `--json` for CI, `--all` for directories. *(#77)*+- **📊 GraphML & CSV Export** — desktop graph tools and spreadsheets; hypergraphs get a hyperedges table. *(#85)*+- **🌐 OrcaRouter Provider** — one key for 150+ models via `create_client("orcarouter")`. *(#71)*+- **🔐 Config File Permissions** — `~/.he/config.toml` saved `0600`. *(#86)*+- **🔗 Obsidian wikilink fix** — aliases no longer break on `[ ] | # ^`. *(#87)*+- **🛡️ Chunk-Level Fault Isolation** — one failed chunk no longer discards the rest of a multi-chunk extraction. *(#78)*+- **⚡ MCP Python SDK 2.x** — `he-mcp` works on mcp 1.x and 2.x. *(#72, #82)*+- **🔀 Directed-Edge Fix** — `(source, target)` order preserved; custom endpoint field names. *(#74)*+- **🔑 DeepSeek API Key Fix** — `DEEPSEEK_API_KEY` honored on the OpenAI-compatible path. *(#76)*+- **🎓 Education Templates** — `course_concept_graph` + `curriculum_structure`. *(#80)*+- **🧭 Smaller Fixes** — Graph_RAG.search 3-tuple; `he talk -i --top-k`; onboarding/docs overhaul. *(#70, #73, #57)*diff --git a/docs/zh/news.md b/docs/zh/news.mdnew file mode 100644index 0000000..46c4dff--- /dev/null+++ b/docs/zh/news.md@@ -0,0 +1,46 @@+# 新闻动态++版本发布与功能亮点。完整变更日志见 [GitHub Releases](https://github.com/yifanfeng97/hyper-extract/releases)。++---++## v0.9.0 — 丰富的文档输入与 chunk_rag 基线++- **📄 丰富的文档输入** — `he parse` / `he feed` 现在支持 PDF、Word、PowerPoint、Excel、HTML、CSV/JSON/XML、EPUB 等格式,安装可选依赖即可启用(`pip install "hyperextract[ingest]"`,由 [MarkItDown](https://github.com/microsoft/markitdown) 驱动)。非 UTF-8 文本(GBK 等)自动识别编码;无文字层的扫描版 PDF 会给出明确的 OCR 提示,而不是静默摄入乱码。+- **🧱 `chunk_rag` 基线方法** — 全新的零提取方法:文档直接分块并向量化,检索返回原始文本块。摄入零 LLM 成本,且具备完整溯源能力(标签、范围检索、按文档回滚)。语料问答与方法对比的块检索基线。+- **🐛 修复** — `he tag` 在所有知识库上都会崩溃(`tag_source` 此前从未在任何类型上实现);`he search`/`he talk`/`he feed`/`he remove --document` 在方法型知识库上会崩溃(`method/*` 模板无法从元数据解析)。++---++## v0.8.1 / v0.8.2++- **🏷️ 来源标签与范围检索** — `he tag ./ka/ --source doc-1 --add legal`,然后 `he search ./ka/ "query" --tag legal` 即可只在已标注文档范围内检索。适用于 graph、hypergraph 和 set。*(#89, #84)*+- **🛡️ 输入类型校验** — `he parse` / `he feed` 现在会拒绝不支持的文件类型并给出转换提示,而不是静默摄入乱码。*(#88)*+- **📦 文档存档修复** — 同一来源换文件名重喂不再积累陈旧副本。*(#89)*++---++## v0.8.0++- **🔄 文档级 Upsert** — 对已标注的文档重新喂入时,旧版本贡献自动回滚:删除的事实消失,共享键从幸存来源重新合并。*(#84)*+- **📁 逐文件来源标注** — `he parse ./docs/` 自动按文件名标注来源;之后可回滚或审计任意单个文件。显式 `--source` 仍然优先。+- **⏱️ 时空溯源** — 时间/空间/时空图谱完整支持来源标注与回滚,并通过确定性的(MERGE_FIELD)回放测试。++---++## v0.5.0 – v0.7.0++- **🗑️ 两级知识删除** — 按键硬删(`he remove --node/--edge`,孤立边自动修剪),或通过 LLM 辅助编辑删除单条错误事实(`he remove --edit-node --fact`),支持 dry-run、键不变性检查与自动备份。*(#84)*+- **📜 来源标注与溯源** — `he feed --source` / `he parse --source` 记录每个文档的原始贡献;`he remove --document` 精确回滚单个文档的贡献;`he info --sources` 展示来源账本。*(#84)*+- **📈 全面增量** — feed/parse/删除/编辑均原地修补向量索引(仅重嵌入受影响的向量);内容哈希未变的文档自动跳过(`--refeed` 强制)。*(#84)*+- **🧪 `he template validate`** — 在为 LLM 调用付费之前发现模板语义错误:9 条诊断规则、`--json` 输出接入 CI、`--all` 扫描目录。*(#77)*+- **📊 GraphML 与 CSV 导出** — 对接桌面图工具与表格软件;超图导出超边表。*(#85)*+- **🌐 OrcaRouter Provider** — 一个 key 接入 150+ 模型(`create_client("orcarouter")`)。*(#71)*+- **🔐 配置文件权限** — `~/.he/config.toml` 以 `0600` 权限保存。*(#86)*+- **🔗 Obsidian 双链修复** — 别名不再因 `[ ] | # ^` 等字符出错。*(#87)*+- **🛡️ Chunk 级故障隔离** — 单个 chunk 失败不再丢弃整个多 chunk 提取结果。*(#78)*+- **⚡ MCP Python SDK 2.x** — `he-mcp` 同时兼容 mcp 1.x 与 2.x。*(#72, #82)*+- **🔀 有向边修复** — 保留 `(source, target)` 顺序;支持自定义端点字段名。*(#74)*+- **🔑 DeepSeek API Key 修复** — OpenAI 兼容路径下正确识别 `DEEPSEEK_API_KEY`。*(#76)*+- **🎓 教育领域模板** — `course_concept_graph` + `curriculum_structure`。*(#80)*+- **🧭 其他修复** — Graph_RAG.search 三元组;`he talk -i --top-k`;引导与文档整体翻新。*(#70, #73, #57)*diff --git a/mkdocs.yml b/mkdocs.ymlindex ad62ff9..2cc2aa7 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -156,6 +156,7 @@ plugins:- resources/faq.md- resources/troubleshooting.md- resources/contributing.md+ - News: news.md- locale: zhname: 中文build: true@@ -256,6 +257,7 @@ plugins:- zh/resources/faq.md- zh/resources/troubleshooting.md- zh/resources/contributing.md+ - 新闻动态: zh/news.mdextra:social:diff --git a/uv.lock b/uv.lockindex d156bb5..125854c 100644--- a/uv.lock+++ b/uv.lock@@ -977,7 +977,7 @@ wheels = [[[package]]name = "hyperextract"-version = "0.8.2"+version = "0.9.0"source = { editable = "." }dependencies = [{ name = "faiss-cpu" },