Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
docs: refresh provenance roadmap and FAQ for v0.9.0 reality
yifanfeng97f9634fc05c42
diff --git a/docs/en/python/guides/provenance.md b/docs/en/python/guides/provenance.mdindex b287de9..e9130e0 100644--- a/docs/en/python/guides/provenance.md+++ b/docs/en/python/guides/provenance.md@@ -146,17 +146,22 @@ See [`he remove`](../../cli/commands/remove.md) for all guardrails (key invarian├── data.json # Extracted knowledge├── metadata.json # Extraction metadata├── sources_nodes.json # Node ledger: raw items per source-└── sources_edges.json # Edge ledger: raw items per source+├── sources_edges.json # Edge ledger: raw items per source+├── sources_chunks.json # Chunk ledger (chunk-based KAs): raw chunks per source+└── documents/ # Archived originals, one current copy per source```-- Keep these files together with the KA (treat them like `data.json` in version control): the raw per-source results live only there, so deleting the ledger disables rollback and change detection for that KA.+- Keep the ledger files together with the KA (treat them like `data.json` in version control): the raw per-source results live only there, so deleting the ledger disables rollback and change detection for that KA.+- `he feed --source` also archives the original document under `documents/` (raw bytes, one current copy per source; disable with `--no-store-doc`). Pass `--purge-documents` to `he remove --document` to delete the archived copy together with the rollback.- Before any write, `he remove` backs up `data.json` as `data.json.bak.<timestamp>` (unless `--no-backup`), so even a rollback can itself be undone.---## Roadmap-Content-hash change detection ships today: `he feed --source` hashes the input and skips unchanged documents before any LLM call. Deeper lineage — mapping individual facts back to the exact chunk (passage) they were extracted from — is planned.+- **Shipped (v0.8.0)** — content-hash change detection: `he feed --source` hashes the input and skips unchanged documents before any LLM call.+- **Shipped (v0.9.0)** — original-document archiving (`documents/`) and the `chunk_rag` method, which retrieves raw text chunks directly (chunk-level *retrieval*).+- **Planned** — deeper lineage for extracted facts: mapping each node/edge in a graph KA back to the exact chunk (passage) it was extracted from. Note this is distinct from `chunk_rag`, which retrieves chunks but does not structure them.---diff --git a/docs/en/resources/faq.md b/docs/en/resources/faq.mdindex 386da1d..99a4e39 100644--- a/docs/en/resources/faq.md+++ b/docs/en/resources/faq.md@@ -125,12 +125,14 @@ he list template### How do I process a PDF?-Convert to text first:+Feed it directly (requires the optional ingest extra — also supports Word, PowerPoint, Excel, HTML, EPUB and more):```bash-pdftotext document.pdf document.txt-he parse document.txt -t general/graph -l en+pip install "hyperextract[ingest]"+he parse document.pdf -t general/graph -o ./ka/ -l en```+Scanned (image-only) PDFs have no text layer — run OCR first. See [he parse](../cli/commands/parse.md) for the full format table.+### Can I process multiple documents?**Option 1**: Feed incrementallydiff --git a/docs/zh/python/guides/provenance.md b/docs/zh/python/guides/provenance.mdindex 5c04adc..8830d0b 100644--- a/docs/zh/python/guides/provenance.md+++ b/docs/zh/python/guides/provenance.md@@ -146,17 +146,22 @@ he remove ./ka/ --edit-node Apple --fact "founded by Steve Jobs" -y├── data.json # 提取的知识├── metadata.json # 提取元数据├── sources_nodes.json # 节点台账:每个来源的原始条目-└── sources_edges.json # 边台账:每个来源的原始条目+├── sources_edges.json # 边台账:每个来源的原始条目+├── sources_chunks.json # 块台账(chunk 型知识库):每个来源的原始块+└── documents/ # 原始文档归档,每个来源保留一份当前副本```-- 请将这些文件与知识摘要放在一起保存(像 `data.json` 一样纳入版本控制):每个来源的原始结果只存在于这里,删除台账文件会导致该知识摘要无法回滚、也无法做变更检测。+- 请将台账文件与知识摘要放在一起保存(像 `data.json` 一样纳入版本控制):每个来源的原始结果只存在于这里,删除台账文件会导致该知识摘要无法回滚、也无法做变更检测。+- `he feed --source` 会同时把原始文档归档到 `documents/`(保留原始字节,每个来源一份当前副本;用 `--no-store-doc` 关闭)。`he remove --document` 时传入 `--purge-documents` 可在回滚的同时删除归档副本。- 每次写入前,`he remove` 会将 `data.json` 备份为 `data.json.bak.<时间戳>`(除非传入 `--no-backup`),因此回滚本身也可以撤销。---## 路线图-基于内容哈希的变更检测已随本次发布落地:`he feed --source` 会在任何 LLM 调用之前计算输入哈希并跳过未变更文档。更细粒度的溯源——把单条事实映射回它被提取自的确切文本块(chunk)——已在规划中。+- **已落地(v0.8.0)** —— 基于内容哈希的变更检测:`he feed --source` 会在任何 LLM 调用之前计算输入哈希并跳过未变更文档。+- **已落地(v0.9.0)** —— 原始文档归档(`documents/`),以及直接检索原始文本块的 `chunk_rag` 方法(chunk 级*检索*)。+- **规划中** —— 更细粒度的溯源:把图谱型知识库中的每个节点/边映射回它被提取自的确切文本块(chunk)。注意这与 `chunk_rag` 不同——后者检索 chunk,但不做结构化。---diff --git a/docs/zh/resources/faq.md b/docs/zh/resources/faq.mdindex feb711e..b357b60 100644--- a/docs/zh/resources/faq.md+++ b/docs/zh/resources/faq.md@@ -125,12 +125,14 @@ he list template### 如何处理 PDF 文件?-先转换为文本:+直接喂入即可(需要可选的 ingest extra——同时支持 Word、PowerPoint、Excel、HTML、EPUB 等):```bash-pdftotext document.pdf document.txt-he parse document.txt -t general/graph -l zh+pip install "hyperextract[ingest]"+he parse document.pdf -t general/graph -o ./ka/ -l zh```+扫描版(纯图片)PDF 没有文字层——请先做 OCR。完整格式表见 [he parse](../cli/commands/parse.md)。+### 可以处理多个文档吗?**选项 1**:增量添加