Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: honor --source/--tag in he talk and ka.chat
Frankie-Xu3ac35b058a2f
diff --git a/docs/en/cli/commands/talk.md b/docs/en/cli/commands/talk.mdindex 258caa7..f922092 100644--- a/docs/en/cli/commands/talk.md+++ b/docs/en/cli/commands/talk.md@@ -23,6 +23,8 @@ he talk KA_PATH [OPTIONS]| `--query` | `-q` | Single question to ask || `--top-k` | `-n` | Number of context items to retrieve | 3 || `--interactive` | `-i` | Enter interactive chat mode |+| `--source` | — | Scope: only knowledge contributed by these source documents (repeatable) |+| `--tag` | — | Scope: only knowledge from sources carrying these tags (repeatable) |---@@ -117,6 +119,24 @@ he talk ./output/ -q "Explain the War of Currents" -n 10---+## Scoped chat++Restrict generated answers to a subset of sources with `--source` (specific source documents) or `--tag` (any source carrying the tag — set tags with [`he tag`](tag.md)), the same filters as [`he search`](search.md):++```bash+# Only answer from sources tagged `legal`+he talk ./ka/ -q "What are the termination conditions?" --tag legal++# Only answer from two specific documents+he talk ./ka/ -q "What are the termination conditions?" --source contract-2024 --source annex-b+```++`--source` and `--tag` combine as a union: a key matches when **any** of its contributing sources fits either filter. Interactive mode (`-i`) keeps the same scope on every turn.++> Scoping requires the KA to have been fed with `--source`, so the source ledger exists — see [Source Attribution & Provenance](../../python/guides/provenance.md). Types without a ledger (for example AutoList / AutoModel) reject `--source` / `--tag` with a scoped-chat error.++---+## Interactive CommandsIn interactive mode (`-i`):diff --git a/docs/en/python/guides/search-and-chat.md b/docs/en/python/guides/search-and-chat.mdindex e430d44..0e1b33d 100644--- a/docs/en/python/guides/search-and-chat.md+++ b/docs/en/python/guides/search-and-chat.md@@ -119,6 +119,18 @@ response = result.chat()```+### Scoped chat++`ka.chat()` accepts the same optional `source_ids` / `tags` as `search()`. Graph types forward them to `search_nodes` / `search_edges`. Types whose `search()` has those names (document / set / graph family) honor the scope; AutoList / AutoModel ignore them instead of raising `TypeError`.++```python+response = result.chat(+ "What are the termination conditions?",+ source_ids=["contract-2024"],+ tags=["legal"],+)+```+### Chat Use Cases```pythondiff --git a/docs/zh/cli/commands/talk.md b/docs/zh/cli/commands/talk.mdindex f6dbc50..8a8356c 100644--- a/docs/zh/cli/commands/talk.md+++ b/docs/zh/cli/commands/talk.md@@ -23,6 +23,8 @@ he talk KA_PATH [OPTIONS]| `--query` | `-q` | 要问的单个问题 || `--top-k` | `-n` | 要检索的上下文项目数量 | 3 || `--interactive` | `-i` | 进入交互式聊天模式 |+| `--source` | — | 范围限定:仅使用这些来源文档贡献的知识(可重复) |+| `--tag` | — | 范围限定:仅使用带有这些标签的来源贡献的知识(可重复) |---@@ -113,6 +115,24 @@ he talk ./output/ -q "解释电流战争" -n 10---+## 范围限定对话++用 `--source`(指定来源文档)或 `--tag`(带有该标签的任意来源 —— 标签用 [`he tag`](tag.md) 管理)把生成答案限定到部分来源,过滤条件与 [`he search`](search.md) 相同:++```bash+# 只根据带有 `legal` 标签的来源作答+he talk ./ka/ -q "终止条件是什么?" --tag legal++# 只根据两份指定文档作答+he talk ./ka/ -q "终止条件是什么?" --source contract-2024 --source annex-b+```++`--source` 与 `--tag` 以并集方式组合:只要某个键的**任一**贡献来源满足任一过滤条件,该键即匹配。交互模式(`-i`)每一轮都沿用同一范围。++> 范围限定要求知识库在摄入时带有 `--source`,这样来源台账才会存在 —— 见[来源标注与溯源](../../python/guides/provenance.md)。没有台账的类型(例如 AutoList / AutoModel)遇到 `--source` / `--tag` 会以 scoped-chat 错误退出。++---+## 交互命令在交互模式(`-i`)中:diff --git a/docs/zh/python/guides/search-and-chat.md b/docs/zh/python/guides/search-and-chat.mdindex 9000310..96a4bb5 100644--- a/docs/zh/python/guides/search-and-chat.md+++ b/docs/zh/python/guides/search-and-chat.md@@ -119,6 +119,18 @@ response = result.chat()```+### 范围限定对话++`ka.chat()` 接受与 `search()` 相同的可选参数 `source_ids` / `tags`。图谱类型会把它们转发给 `search_nodes` / `search_edges`。`search()` 签名里带这两个名字的类型(document / set / 图谱家族)会遵守范围;AutoList / AutoModel 会忽略它们,而不是抛出 `TypeError`。++```python+response = result.chat(+ "终止条件是什么?",+ source_ids=["contract-2024"],+ tags=["legal"],+)+```+### 聊天用例```pythondiff --git a/hyperextract/cli/cli.py b/hyperextract/cli/cli.pyindex fffdea3..4eef923 100644--- a/hyperextract/cli/cli.py+++ b/hyperextract/cli/cli.py@@ -1081,7 +1081,13 @@ def search()-def chat_loop(ka, ka_path: str, top_k: int = 3):+def chat_loop(+ ka,+ ka_path: str,+ top_k: int = 3,+ source_ids: list[str] | None = None,+ tags: list[str] | None = None,+):"""Interactive chat loop."""console.print("\n[bold green]Entering interactive mode. Type 'exit' or 'quit' to stop.[/bold green]\n"@@ -1103,7 +1109,12 @@ def chat_loop(ka, ka_path: str, top_k: int = 3):breakif not query.strip():continue- response = ka.chat(query, top_k=top_k)+ chat_kwargs = {"top_k": top_k}+ if source_ids:+ chat_kwargs["source_ids"] = source_ids+ if tags:+ chat_kwargs["tags"] = tags+ response = ka.chat(query, **chat_kwargs)console.print()console.print(response.content)console.print()@@ -1182,6 +1193,12 @@ def talk(interactive: bool = typer.Option(False, "--interactive", "-i", help="Interactive mode"),+ source: list[str] | None = typer.Option(+ None, "--source", help="Scope: only knowledge from these source documents"+ ),+ tag: list[str] | None = typer.Option(+ None, "--tag", help="Scope: only knowledge from sources carrying these tags"+ ),):"""Chat with Knowledge Abstract."""logger.info(@@ -1228,12 +1245,30 @@ def talk(console.print(f"[red]Error:[/red] {e}")raise typer.Exit(1)+ scope_kwargs: dict[str, list[str]] = {}+ if source:+ scope_kwargs["source_ids"] = list(source)+ if tag:+ scope_kwargs["tags"] = list(tag)+ if scope_kwargs:+ import inspect++ chat_params = inspect.signature(type(ka).chat).parameters+ for flag, key in (("--source", "source_ids"), ("--tag", "tags")):+ if key in scope_kwargs and key not in chat_params:+ console.print(+ f"[red]Error:[/red] Scoped chat ({flag}) is not "+ f"supported by {type(ka).__name__} knowledge "+ "abstracts (no source ledger)."+ )+ raise typer.Exit(1)+if interactive:- chat_loop(ka, ka_path, top_k=top_k)+ chat_loop(ka, ka_path, top_k=top_k, **scope_kwargs)else:with console.status("[bold blue]Thinking..."):try:- response = ka.chat(query, top_k=top_k)+ response = ka.chat(query, top_k=top_k, **scope_kwargs)console.print(response.content)if response.additional_kwargs.get("retrieved_items"):diff --git a/hyperextract/types/base.py b/hyperextract/types/base.pyindex ae64cb4..473b096 100644--- a/hyperextract/types/base.py+++ b/hyperextract/types/base.py@@ -544,7 +544,14 @@ class BaseAutoType(ABC, Generic[T]):List of relevant knowledge items."""- def chat(self, query: str, top_k: int = 3) -> AIMessage:+ def chat(+ self,+ query: str,+ top_k: int = 3,+ *,+ source_ids: list[str] | None = None,+ tags: list[str] | None = None,+ ) -> AIMessage:"""Performs a chat-like interaction with the knowledge abstract.This generic method retrieves relevant items and generates a response.@@ -554,12 +561,22 @@ class BaseAutoType(ABC, Generic[T]):Args:query: User query string.top_k: Number of relevant items to retrieve (default: 3).+ source_ids: Optional scope forwarded to search() when the type accepts it.+ tags: Optional scope forwarded to search() when the type accepts it.Returns:An AIMessage object containing the LLM-generated response."""+ import inspect+# Step 1: Retrieve relevant items from knowledge abstract- search_results = self.search(query, top_k)+ search_kwargs: dict[str, Any] = {}+ search_params = inspect.signature(type(self).search).parameters+ if "source_ids" in search_params:+ search_kwargs["source_ids"] = source_ids+ if "tags" in search_params:+ search_kwargs["tags"] = tags+ search_results = self.search(query, top_k, **search_kwargs)# Step 2: Format context from retrieved itemsformatted_context = []diff --git a/hyperextract/types/graph.py b/hyperextract/types/graph.pyindex d38e027..3e819df 100644--- a/hyperextract/types/graph.py+++ b/hyperextract/types/graph.py@@ -1244,6 +1244,9 @@ class AutoGraph(top_k: int | None = None,top_k_nodes: int = 3,top_k_edges: int = 3,+ *,+ source_ids: list[str] | None = None,+ tags: list[str] | None = None,) -> AIMessage:"""Performs a chat-like interaction using graph knowledge.@@ -1256,6 +1259,8 @@ class AutoGraph(If provided, sets both top_k_nodes and top_k_edges to this value.top_k_nodes: Number of relevant nodes to retrieve (default: 3). Set to 0 to disable node context.top_k_edges: Number of relevant edges to retrieve (default: 3). Set to 0 to disable edge context.+ source_ids: Optional scope — only nodes/edges contributed by these source documents.+ tags: Optional scope — only nodes/edges from sources carrying these tags.Returns:An AIMessage object containing the LLM-generated response.@@ -1280,7 +1285,9 @@ class AutoGraph(# Step 2: Retrieve and format nodes contextnodes = []if top_k_nodes > 0:- nodes = self.search_nodes(query, top_k=top_k_nodes)+ nodes = self.search_nodes(+ query, top_k=top_k_nodes, source_ids=source_ids, tags=tags+ )if nodes:context_parts.append("=== Relevant Nodes ===")for node in nodes:@@ -1292,7 +1299,9 @@ class AutoGraph(# Step 3: Retrieve and format edges contextedges = []if top_k_edges > 0:- edges = self.search_edges(query, top_k=top_k_edges)+ edges = self.search_edges(+ query, top_k=top_k_edges, source_ids=source_ids, tags=tags+ )if edges:context_parts.append("=== Relevant Edges ===")for edge in edges:diff --git a/hyperextract/types/hypergraph.py b/hyperextract/types/hypergraph.pyindex f62ec81..5926b1b 100644--- a/hyperextract/types/hypergraph.py+++ b/hyperextract/types/hypergraph.py@@ -654,6 +654,9 @@ class AutoHypergraph(top_k_nodes: int = 3,top_k_edges: int = 3,top_k: int | None = None,+ *,+ source_ids: list[str] | None = None,+ tags: list[str] | None = None,) -> AIMessage:"""Performs a chat-like interaction using hypergraph knowledge.@@ -665,6 +668,8 @@ class AutoHypergraph(top_k_nodes: Number of relevant nodes to retrieve (default: 3). Set to 0 to disable node context.top_k_edges: Number of relevant hyperedges to retrieve (default: 3). Set to 0 to disable edge context.top_k: If provided, sets both top_k_nodes and top_k_edges to this value.+ source_ids: Optional scope — only nodes/edges contributed by these source documents.+ tags: Optional scope — only nodes/edges from sources carrying these tags.Returns:An AIMessage object containing the LLM-generated response.@@ -689,7 +694,9 @@ class AutoHypergraph(# Step 2: Retrieve and format nodes contextnodes = []if top_k_nodes > 0:- nodes = self.search_nodes(query, top_k=top_k_nodes)+ nodes = self.search_nodes(+ query, top_k=top_k_nodes, source_ids=source_ids, tags=tags+ )if nodes:context_parts.append("=== Relevant Nodes ===")for node in nodes:@@ -701,7 +708,9 @@ class AutoHypergraph(# Step 3: Retrieve and format edges contextedges = []if top_k_edges > 0:- edges = self.search_edges(query, top_k=top_k_edges)+ edges = self.search_edges(+ query, top_k=top_k_edges, source_ids=source_ids, tags=tags+ )if edges:context_parts.append("=== Relevant Edges ===")for edge in edges:diff --git a/tests/cli/test_talk.py b/tests/cli/test_talk.pyindex eb19cc6..8b55257 100644--- a/tests/cli/test_talk.py+++ b/tests/cli/test_talk.py@@ -1,6 +1,29 @@-"""Tests for the interactive `he talk` chat loop."""+"""Tests for the interactive `he talk` chat loop and scoped chat flags."""++import json+from pathlib import Path+from unittest.mock import patch++from typer.testing import CliRunnerimport hyperextract.cli.cli as climod+from hyperextract.cli.cli import app++runner = CliRunner()+++def _ka_dir(tmp_path: Path) -> Path:+ ka = tmp_path / "ka"+ ka.mkdir()+ (ka / "data.json").write_text("{}", encoding="utf-8")+ (ka / "metadata.json").write_text(+ json.dumps({"template": "general/graph", "lang": "en"}),+ encoding="utf-8",+ )+ index = ka / "index"+ index.mkdir()+ (index / "dummy").write_text("x", encoding="utf-8")+ return kadef test_chat_loop_passes_top_k(monkeypatch):@@ -8,13 +31,78 @@ def test_chat_loop_passes_top_k(monkeypatch):recorded = {}class _StubKA:- def chat(self, query, top_k=3):+ def chat(self, query, top_k=3, *, source_ids=None, tags=None):recorded["top_k"] = top_k+ recorded["source_ids"] = source_ids+ recorded["tags"] = tagsreturn type("_Resp", (), {"content": "ok"})()queries = iter(["hello", "exit"])monkeypatch.setattr(climod.console, "input", lambda *a, **k: next(queries))- climod.chat_loop(_StubKA(), "some/ka", top_k=10)+ climod.chat_loop(_StubKA(), "some/ka", top_k=10, source_ids=["s1"], tags=["t1"])assert recorded["top_k"] == 10+ assert recorded["source_ids"] == ["s1"]+ assert recorded["tags"] == ["t1"]+++def test_talk_forwards_source_and_tag(tmp_path):+ recorded = {}++ class _GraphChatKA:+ def load(self, path):+ pass++ def chat(self, query, top_k=3, *, source_ids=None, tags=None):+ recorded["query"] = query+ recorded["source_ids"] = source_ids+ recorded["tags"] = tags+ return type("_Resp", (), {"content": "ok", "additional_kwargs": {}})()++ ka_dir = _ka_dir(tmp_path)+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch("hyperextract.cli.cli.Template.create", return_value=_GraphChatKA()),+ ):+ result = runner.invoke(+ app,+ [+ "talk",+ str(ka_dir),+ "-q",+ "hello",+ "--source",+ "s1",+ "--tag",+ "t1",+ ],+ )++ assert result.exit_code == 0, result.output+ assert recorded["query"] == "hello"+ assert recorded["source_ids"] == ["s1"]+ assert recorded["tags"] == ["t1"]+++def test_talk_scope_rejected_without_ledger(tmp_path):+ class _ListChatKA:+ def load(self, path):+ pass++ def chat(self, query, top_k=3):+ return type("_Resp", (), {"content": "ok", "additional_kwargs": {}})()++ ka_dir = _ka_dir(tmp_path)+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch("hyperextract.cli.cli.Template.create", return_value=_ListChatKA()),+ ):+ result = runner.invoke(+ app,+ ["talk", str(ka_dir), "-q", "hello", "--source", "s1"],+ )++ assert result.exit_code == 1+ assert "source ledger" in result.output+ assert "scoped chat" in result.output.lower() or "Scoped chat" in result.outputdiff --git a/tests/types/test_document.py b/tests/types/test_document.pyindex 91b6a88..48839ca 100644--- a/tests/types/test_document.py+++ b/tests/types/test_document.py@@ -82,6 +82,23 @@ class TestSearch:# The untagged source is excluded from the same query.assert all("Alpha" not in c.content for c in results)+ def test_chat_forwards_scope_to_search(self, ka):+ from langchain_core.messages import AIMessage+ from langchain_core.runnables import RunnableLambda++ seen = {}++ def _spy(query, top_k=3, **kwargs):+ seen["source_ids"] = kwargs.get("source_ids")+ seen["tags"] = kwargs.get("tags")+ return []++ ka.search = _spy+ ka.llm_client = RunnableLambda(lambda _: AIMessage(content="ok"))+ ka.chat("q", source_ids=["s1"], tags=["t1"])+ assert seen["source_ids"] == ["s1"]+ assert seen["tags"] == ["t1"]+class TestUpsertAndRollback:def test_upsert_replaces_old_chunks(self, ka):diff --git a/tests/types/test_graph_chat_scope.py b/tests/types/test_graph_chat_scope.pynew file mode 100644index 0000000..063e1d2--- /dev/null+++ b/tests/types/test_graph_chat_scope.py@@ -0,0 +1,95 @@+"""AutoGraph / AutoHypergraph.chat must forward source_ids/tags to search helpers."""++from langchain_core.messages import AIMessage+from langchain_core.runnables import RunnableLambda+from pydantic import BaseModel++from hyperextract.types import AutoGraph, AutoHypergraph+++class _Entity(BaseModel):+ name: str+ type: str = "x"+++class _Relation(BaseModel):+ source: str+ target: str+ relation_type: str = "related"+++class _HyperRelation(BaseModel):+ participants: list[str]+ relation_type: str+++def _graph(llm_client, embedder):+ return AutoGraph(+ node_schema=_Entity,+ edge_schema=_Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=llm_client,+ embedder=embedder,+ )+++def _hypergraph(llm_client, embedder):+ return AutoHypergraph(+ node_schema=_Entity,+ edge_schema=_HyperRelation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.relation_type}_{sorted(x.participants)}",+ nodes_in_edge_extractor=lambda x: tuple(x.participants),+ llm_client=llm_client,+ embedder=embedder,+ )+++def test_autograph_chat_forwards_scope(llm_client, embedder):+ ka = _graph(llm_client, embedder)+ seen = {}++ def _nodes(query, **kw):+ seen["nodes"] = kw+ return []++ def _edges(query, **kw):+ seen["edges"] = kw+ return []++ ka.search_nodes = _nodes+ ka.search_edges = _edges+ ka.llm_client = RunnableLambda(lambda _: AIMessage(content="ok"))++ ka.chat("q", source_ids=["s1"], tags=["t1"])++ assert seen["nodes"]["source_ids"] == ["s1"]+ assert seen["nodes"]["tags"] == ["t1"]+ assert seen["edges"]["source_ids"] == ["s1"]+ assert seen["edges"]["tags"] == ["t1"]+++def test_autohypergraph_chat_forwards_scope(llm_client, embedder):+ ka = _hypergraph(llm_client, embedder)+ seen = {}++ def _nodes(query, **kw):+ seen["nodes"] = kw+ return []++ def _edges(query, **kw):+ seen["edges"] = kw+ return []++ ka.search_nodes = _nodes+ ka.search_edges = _edges+ ka.llm_client = RunnableLambda(lambda _: AIMessage(content="ok"))++ ka.chat("q", source_ids=["s1"], tags=["t1"])++ assert seen["nodes"]["source_ids"] == ["s1"]+ assert seen["nodes"]["tags"] == ["t1"]+ assert seen["edges"]["source_ids"] == ["s1"]+ assert seen["edges"]["tags"] == ["t1"]diff --git a/tests/types/test_set_provenance.py b/tests/types/test_set_provenance.pyindex ee19e4e..a664a53 100644--- a/tests/types/test_set_provenance.py+++ b/tests/types/test_set_provenance.py@@ -64,6 +64,23 @@ class TestSourceRecording:other.build_index()assert other.search("Apple", top_k=5, source_ids=["d1"]) == []+ def test_chat_forwards_scope_to_search(self, ka):+ from langchain_core.messages import AIMessage+ from langchain_core.runnables import RunnableLambda++ seen = {}++ def _spy(query, top_k=3, **kwargs):+ seen["source_ids"] = kwargs.get("source_ids")+ seen["tags"] = kwargs.get("tags")+ return []++ ka.search = _spy+ ka.llm_client = RunnableLambda(lambda _: AIMessage(content="ok"))+ ka.chat("q", source_ids=["s1"], tags=["t1"])+ assert seen["source_ids"] == ["s1"]+ assert seen["tags"] == ["t1"]+class TestTagsAndRollback:def test_tag_round_trip(self, ka):