Skip to content

Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.

docs: add Managing Documents Over Time workflow guide (EN+ZH)

yifanfeng97e7aad38423e3
diff --git a/docs/en/cli/commands/feed.md b/docs/en/cli/commands/feed.md
index d0e9fed..f72e5d9 100644
--- a/docs/en/cli/commands/feed.md
+++ b/docs/en/cli/commands/feed.md
@@ -203,3 +203,9 @@ he feed ./ka/ doc.md -t general/biography_graph
- [`he parse`](parse.md) — Create new knowledge abstract
- [`he info`](info.md) — View knowledge abstract statistics
- [`he build-index`](build-index.md) — Rebuild search index after feeding
+
+## See Also
+
+- [`he remove --document`](remove.md) — roll back a whole source document
+- [`he tag`](tag.md) — tag sources for scoped search
+- [Managing Documents Over Time](managing-documents.md) — the complete lifecycle guide
diff --git a/docs/en/cli/commands/managing-documents.md b/docs/en/cli/commands/managing-documents.md
new file mode 100644
index 0000000..1b3a5fa
--- /dev/null
+++ b/docs/en/cli/commands/managing-documents.md
@@ -0,0 +1,189 @@
+# Managing Documents Over Time
+
+A knowledge abstract is not a one-shot artifact — it evolves as your source documents change. This guide walks through the complete lifecycle: **ingest → attribute → tag → search within scope → update → roll back → audit**.
+
+---
+
+## Prerequisites
+
+- A graph-family knowledge abstract (graph / hypergraph / temporal / spatial)
+- hyperextract installed (`pip install hyperextract`)
+- An LLM provider configured (`he config init ...`)
+
+---
+
+## The Workflow
+
+```mermaid
+flowchart LR
+ A[📄 Feed + Attribute] --> B[🏷️ Tag]
+ B --> C[🔎 Scoped Search]
+ C --> D[🔄 Update / Upsert]
+ D --> E[↩️ Roll Back]
+ E --> F[👁️ Audit]
+
+ style A fill:#e8f5e9
+ style E fill:#ffebee
+ style F fill:#e3f2fd
+```
+
+### 1. Ingest with Attribution
+
+Every document you feed can carry a `source_id` — the foundation for per-document rollback and change detection:
+
+```bash
+he feed ./ka/ contract-acme.md --source contract-acme
+he feed ./ka/ contract-globex.md --source contract-globex
+he parse ./more-docs/ -o ./ka/ -t legal/contract_obligation -l en --source more-docs
+```
+
+```python
+ka.feed_text(text, source_id="contract-acme")
+```
+
+!!! tip
+ When the input is a **directory**, `he parse` automatically attributes each file by its stem — no explicit `--source` needed.
+
+### 2. Tag Sources
+
+Group documents with tags so you can scope searches and batch-identify groups later:
+
+```bash
+he tag ./ka/ --source contract-acme --add legal --add acme --add reviewed
+he tag ./ka/ --source contract-globex --add legal --add globex
+he tag ./ka/ --list
+```
+
+### 3. Search Within Scope
+
+Restrict retrieval to a subset of documents by source or tag:
+
+```bash
+# Only knowledge from legal-tagged documents
+he search ./ka/ "termination clause" --tag legal
+
+# Only knowledge from a specific document
+he search ./ka/ "payment terms" --source contract-acme
+
+# Union: match either filter
+he search ./ka/ "liability" --tag legal --source contract-globex
+```
+
+### 4. Update a Document (Upsert)
+
+When a source document changes, simply re-feed it under the same source id. The previous version's contributions are rolled back automatically — facts removed from the updated document disappear, and shared keys re-merge from surviving sources:
+
+```bash
+he feed ./ka/ contract-acme-v2.md --source contract-acme
+```
+
+Facts removed from the updated document **do not linger**.
+
+### 5. Roll Back a Document
+
+Remove every contribution of one document without touching the rest:
+
+```bash
+he remove ./ka/ --document contract-globex
+```
+
+Keys shared with surviving documents are re-merged from their raw results — deterministic with classic merge strategies. For a scorched-earth approach, use `--strategy touched` to delete every key the source touched.
+
+You can also remove individual items:
+
+```bash
+he remove ./ka/ --node Apple
+he remove ./ka/ --edit-node Apple --fact "founded by Steve Jobs"
+```
+
+### 6. Audit the Ledger
+
+See which documents contributed to your knowledge abstract and how much:
+
+```bash
+he info ./ka/ --sources
+```
+
+```
+┌─────────────────────── Source Ledger ───────────────────────┐
+│ Source ID │ Raw Items │ Content Hash │ Tags │
+├───────────────────┼───────────┼──────────────┼─────────────┤
+│ contract-acme │ 3 │ a1b2c3d4... │ legal,acme │
+│ contract-globex │ 2 │ e5f6a7b8... │ legal │
+└───────────────────┴───────────┴──────────────┴─────────────┘
+```
+
+---
+
+## Persistence
+
+The KA directory is **self-contained**:
+
+```
+ka/
+├── data.json # merged knowledge (the derived view)
+├── metadata.json # template, lang, timestamps
+├── sources_nodes.json # node source ledger (raw items + hashes + tags)
+├── sources_edges.json # edge source ledger
+├── documents/ # archived source documents
+│ ├── a1b2c3d4e5f6-contract-acme.md
+│ └── e5f6a7b8c9d0-contract-globex.md
+└── index/ # vector index (patched in place)
+```
+
+Copy the directory and you have the complete knowledge **plus** the full evidence trail.
+
+---
+
+## Change Detection
+
+`he feed --source` computes a SHA-256 content hash and records it in the ledger. Re-feeding an unchanged document is skipped automatically:
+
+```
+Source 'contract-acme' is unchanged (content hash matches) — nothing to do.
+Use --refeed to re-ingest anyway.
+```
+
+Use `--refeed` to force re-ingestion (e.g. after upgrading the extraction template or LLM model).
+
+---
+
+## Python API
+
+```python
+from hyperextract.types import AutoGraph
+
+ka = AutoGraph(node_schema=..., edge_schema=..., ...)
+
+# Attributed ingestion
+ka.feed_text(text, source_id="contract-acme")
+
+# Tag
+ka.tag_source("contract-acme", add=["legal", "acme"])
+
+# Scoped search
+nodes, edges = ka.search("termination", tags=["legal"])
+
+# Document upsert (roll back old + merge new)
+ka.feed_text(v2_text, source_id="contract-acme")
+
+# Roll back a document
+report = ka.remove_source("contract-globex")
+
+# Audit
+print(ka.sources())
+
+# Persist (ledger + documents included)
+ka.dump("./ka/")
+```
+
+---
+
+## See Also
+
+- [`he feed`](feed.md) — incremental ingestion
+- [`he remove`](remove.md) — all deletion modes (key / fact / document)
+- [`he tag`](tag.md) — tag management
+- [`he search`](search.md) — scoped search options
+- [`he info --sources`](info.md) — ledger audit
+- [Provenance Guide (Python SDK)](../../python/guides/provenance.md) — the same lifecycle via the Python API
diff --git a/docs/en/cli/commands/remove.md b/docs/en/cli/commands/remove.md
index 04ddeff..b1310c2 100644
--- a/docs/en/cli/commands/remove.md
+++ b/docs/en/cli/commands/remove.md
@@ -140,3 +140,9 @@ Works for graph, hypergraph, and temporal/spatial graph knowledge abstracts.
- [`he clean`](clean.md) — remove the search index or the whole KA
- [`he show`](show.md) — inspect keys before deleting
- [`he build-index`](build-index.md) — rebuild the search index after removal
+
+## See Also
+
+- [`he feed --source`](feed.md) — attributed ingestion (the other half of the lifecycle)
+- [`he tag`](tag.md) — tag sources for scoped search
+- [Managing Documents Over Time](managing-documents.md) — the complete lifecycle guide
diff --git a/docs/en/getting-started/cli-quickstart.md b/docs/en/getting-started/cli-quickstart.md
index 88d42fb..38f629a 100644
--- a/docs/en/getting-started/cli-quickstart.md
+++ b/docs/en/getting-started/cli-quickstart.md
@@ -198,6 +198,17 @@ he show ./output/
---
+## What's Next?
+
+Your knowledge abstract is a living artifact — as your source documents change:
+
+- **Update** a document: `he feed ./ka/ updated-doc.md --source doc-1` (re-feed under the same source to upsert)
+- **Roll back** a document: `he remove ./ka/ --document doc-1`
+- **Tag and scope** your searches: `he tag ./ka/ --source doc-1 --add legal`
+- **Audit** the ledger: `he info ./ka/ --sources`
+
+→ [Managing Documents Over Time](../cli/commands/managing-documents.md) — the complete guide
+
## Complete Workflow
Here's the typical workflow:
diff --git a/docs/zh/cli/commands/feed.md b/docs/zh/cli/commands/feed.md
index 0a432e1..2e2397c 100644
--- a/docs/zh/cli/commands/feed.md
+++ b/docs/zh/cli/commands/feed.md
@@ -203,3 +203,9 @@ he feed ./ka/ doc.md -t general/biography_graph
- [`he parse`](parse.md) — 创建新知识库
- [`he info`](info.md) — 查看知识库统计信息
- [`he build-index`](build-index.md) — Feed 后重建搜索索引
+
+## 另请参阅
+
+- [`he remove --document`](remove.md) — 回滚整份来源文档
+- [`he tag`](tag.md) — 为来源打标签以启用范围检索
+- [管理文档的生命周期](managing-documents.md) — 完整生命周期指南
diff --git a/docs/zh/cli/commands/managing-documents.md b/docs/zh/cli/commands/managing-documents.md
new file mode 100644
index 0000000..9d0220e
--- /dev/null
+++ b/docs/zh/cli/commands/managing-documents.md
@@ -0,0 +1,189 @@
+# 管理文档的生命周期
+
+知识摘要不是一次性产物——它随来源文档的变化而演化。本指南完整介绍文档的生命周期:**摄取 → 标注 → 打标签 → 范围检索 → 更新 → 回滚 → 审计**。
+
+---
+
+## 前置条件
+
+- 图谱类知识摘要(graph / hypergraph / temporal / spatial)
+- 已安装 hyperextract(`pip install hyperextract`)
+- 已配置 LLM 提供商(`he config init ...`)
+
+---
+
+## 完整工作流
+
+```mermaid
+flowchart LR
+ A[📄 喂入 + 标注] --> B[🏷️ 打标签]
+ B --> C[🔎 范围检索]
+ C --> D[🔄 更新 / Upsert]
+ D --> E[↩️ 回滚]
+ E --> F[👁️ 审计]
+
+ style A fill:#e8f5e9
+ style E fill:#ffebee
+ style F fill:#e3f2fd
+```
+
+### 1. 带来源标注摄取
+
+每份文档都可以携带 `source_id`——这是按文档回滚和变更检测的基础:
+
+```bash
+he feed ./ka/ contract-acme.md --source contract-acme
+he feed ./ka/ contract-globex.md --source contract-globex
+he parse ./more-docs/ -o ./ka/ -t legal/contract_obligation -l en --source more-docs
+```
+
+```python
+ka.feed_text(text, source_id="contract-acme")
+```
+
+!!! tip
+ 当输入是**目录**时,`he parse` 会自动按文件名标注每个文件——无需显式 `--source`。
+
+### 2. 打标签
+
+用标签对文档分组,以便后续范围检索和批量识别:
+
+```bash
+he tag ./ka/ --source contract-acme --add legal --add acme --add reviewed
+he tag ./ka/ --source contract-globex --add legal --add globex
+he tag ./ka/ --list
+```
+
+### 3. 范围检索
+
+按来源或标签将检索限定在部分文档范围内:
+
+```bash
+# 只检索 legal 标签文档中的知识
+he search ./ka/ "termination clause" --tag legal
+
+# 只检索特定文档贡献的知识
+he search ./ka/ "payment terms" --source contract-acme
+
+# 并集:命中任一过滤条件即保留
+he search ./ka/ "liability" --tag legal --source contract-globex
+```
+
+### 4. 更新文档(Upsert)
+
+来源文档变化后,用同一个 source id 重新喂入即可。旧版本的贡献会自动回滚——更新后删掉的事实不再残留,共享 key 从幸存来源重新合并:
+
+```bash
+he feed ./ka/ contract-acme-v2.md --source contract-acme
+```
+
+更新后删掉的事实**不会残留**。
+
+### 5. 回滚整份文档
+
+删除某份文档的全部贡献,不影响其余知识:
+
+```bash
+he remove ./ka/ --document contract-globex
+```
+
+与其他文档共享的 key 会从幸存来源的原始结果重新合并——经典合并策略下完全确定。如需"全部删除"可用 `--strategy touched`。
+
+也可以删除单个条目:
+
+```bash
+he remove ./ka/ --node Apple
+he remove ./ka/ --edit-node Apple --fact "founded by Steve Jobs"
+```
+
+### 6. 审计来源账本
+
+查看哪些文档贡献了知识以及贡献量:
+
+```bash
+he info ./ka/ --sources
+```
+
+```
+┌─────────────────────── 来源账本 ───────────────────────────┐
+│ 来源 ID │ 原始条目 │ 内容哈希 │ 标签 │
+├───────────────────┼───────────┼──────────────┼─────────────┤
+│ contract-acme │ 3 │ a1b2c3d4... │ legal,acme │
+│ contract-globex │ 2 │ e5f6a7b8... │ legal │
+└───────────────────┴───────────┴──────────────┴─────────────┘
+```
+
+---
+
+## 持久化
+
+KA 目录是**自包含**的:
+
+```
+ka/
+├── data.json # 合并后的知识(派生视图)
+├── metadata.json # 模板、语言、时间戳
+├── sources_nodes.json # 节点来源账本(原始结果 + 哈希 + 标签)
+├── sources_edges.json # 边来源账本
+├── documents/ # 归档的来源文档
+│ ├── a1b2c3d4e5f6-contract-acme.md
+│ └── e5f6a7b8c9d0-contract-globex.md
+└── index/ # 向量索引(原地修补)
+```
+
+复制整个目录,即可带走完整的知识**和**完整的证据链。
+
+---
+
+## 变更检测
+
+`he feed --source` 会计算输入文本的 SHA-256 哈希并记入账本。重复喂入内容未变的文档会自动跳过:
+
+```
+Source 'contract-acme' is unchanged (content hash matches) — nothing to do.
+Use --refeed to re-ingest anyway.
+```
+
+使用 `--refeed` 可强制重新摄取(例如升级抽取模板或 LLM 模型后)。
+
+---
+
+## Python API
+
+```python
+from hyperextract.types import AutoGraph
+
+ka = AutoGraph(node_schema=..., edge_schema=..., ...)
+
+# 带来源标注摄取
+ka.feed_text(text, source_id="contract-acme")
+
+# 打标签
+ka.tag_source("contract-acme", add=["legal", "acme"])
+
+# 范围检索
+nodes, edges = ka.search("termination", tags=["legal"])
+
+# 文档 upsert(回滚旧版 + 合并新版)
+ka.feed_text(v2_text, source_id="contract-acme")
+
+# 回滚整份文档
+report = ka.remove_source("contract-globex")
+
+# 审计
+print(ka.sources())
+
+# 持久化(含账本和文档)
+ka.dump("./ka/")
+```
+
+---
+
+## 另请参阅
+
+- [`he feed`](feed.md) — 增量摄取
+- [`he remove`](remove.md) — 所有删除模式(按键 / 按事实 / 按文档)
+- [`he tag`](tag.md) — 标签管理
+- [`he search`](search.md) — 范围检索选项
+- [`he info --sources`](info.md) — 来源账本审计
+- [溯源指南(Python SDK)](../../python/guides/provenance.md) — 同一生命周期的 Python API 视角
diff --git a/docs/zh/cli/commands/remove.md b/docs/zh/cli/commands/remove.md
index 153f7ec..4283a8f 100644
--- a/docs/zh/cli/commands/remove.md
+++ b/docs/zh/cli/commands/remove.md
@@ -140,3 +140,9 @@ ka.dump("./ka/") # 持久化
- [`he clean`](clean.md) — 删除搜索索引或整个 KA
- [`he show`](show.md) — 删除前查看 key
- [`he build-index`](build-index.md) — 删除后重建搜索索引
+
+## 另请参阅
+
+- [`he feed --source`](feed.md) — 来源标注摄取(生命周期的另一半)
+- [`he tag`](tag.md) — 为来源打标签以启用范围检索
+- [管理文档的生命周期](managing-documents.md) — 完整生命周期指南
diff --git a/docs/zh/getting-started/cli-quickstart.md b/docs/zh/getting-started/cli-quickstart.md
index 79af322..0714a33 100644
--- a/docs/zh/getting-started/cli-quickstart.md
+++ b/docs/zh/getting-started/cli-quickstart.md
@@ -193,6 +193,17 @@ he show ./output/
---
+## 下一步?
+
+你的知识摘要是活的——随来源文档的变化持续演化:
+
+- **更新**文档:`he feed ./ka/ updated-doc.md --source doc-1`(同 source 重喂即 upsert)
+- **回滚**文档:`he remove ./ka/ --document doc-1`
+- **打标签并范围检索**:`he tag ./ka/ --source doc-1 --add legal`
+- **审计**来源账本:`he info ./ka/ --sources`
+
+→ [管理文档的生命周期](../cli/commands/managing-documents.md) — 完整指南
+
## 完整工作流程
以下是典型的工作流程:
diff --git a/mkdocs.yml b/mkdocs.yml
index e052843..ad62ff9 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -78,6 +78,7 @@ plugins:
- cli/commands/build-index.md
- cli/commands/clean.md
- cli/commands/remove.md
+ - cli/commands/managing-documents.md
- cli/commands/search.md
- cli/commands/tag.md
- cli/commands/talk.md
@@ -176,6 +177,7 @@ plugins:
- zh/cli/commands/build-index.md
- zh/cli/commands/clean.md
- zh/cli/commands/remove.md
+ - zh/cli/commands/managing-documents.md
- zh/cli/commands/search.md
- zh/cli/commands/tag.md
- zh/cli/commands/talk.md