Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: add MCP server (he-mcp) exposing knowledge abstracts
Oguz Vuruskaner67281d99be67
diff --git a/README.md b/README.mdindex f6a90ab..ed78bae 100644--- a/README.md+++ b/README.md@@ -226,6 +226,17 @@ identifiers:| Template Gallery | [80+ presets](./hyperextract/templates/presets/) || Examples | [Working code](./examples/) |+## 🔌 MCP Server++Expose your knowledge abstracts to MCP-capable assistants (Claude Desktop, IDE agents) via the [Model Context Protocol](https://modelcontextprotocol.io) — read + export only.++```bash+pip install 'hyperextract[mcp]'+he-mcp # stdio MCP server+```++Tools: `list_templates`, `info`, `search`, `ask` (RAG), `export_obsidian`. Full guide: [MCP Server docs](https://yifanfeng97.github.io/Hyper-Extract/latest/mcp/).+## 🤝 Contributing & LicenseContributions are welcome! Please submit [Issues](https://github.com/yifanfeng97/hyper-extract/issues) and [PRs](https://github.com/yifanfeng97/hyper-extract/pulls).diff --git a/README_ZH.md b/README_ZH.mdindex a08f684..32dbef5 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -226,6 +226,17 @@ identifiers:| 模板画廊 | [80+ 预设模板](./hyperextract/templates/presets/) || 示例代码 | [可运行示例](./examples/) |+## 🔌 MCP 服务器++通过 [Model Context Protocol](https://modelcontextprotocol.io) 将知识摘要暴露给支持 MCP 的助手(Claude Desktop、IDE 智能体)——只读 + 导出。++```bash+pip install 'hyperextract[mcp]'+he-mcp # stdio MCP 服务器+```++工具:`list_templates`、`info`、`search`、`ask`(RAG)、`export_obsidian`。完整指南:[MCP 服务器文档](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/mcp/)。+## 🤝 参与贡献与协议热烈欢迎社区提交 [Issues](https://github.com/yifanfeng97/hyper-extract/issues) 和 [PRs](https://github.com/yifanfeng97/hyper-extract/pulls)。diff --git a/docs/en/mcp.md b/docs/en/mcp.mdnew file mode 100644index 0000000..13978db--- /dev/null+++ b/docs/en/mcp.md@@ -0,0 +1,67 @@+# MCP Server++Hyper-Extract ships an [MCP](https://modelcontextprotocol.io) server so MCP-capable assistants (Claude Desktop, IDE agents, etc.) can **query and export your Knowledge Abstracts** over the Model Context Protocol.++It is **read + export only** — it never creates, mutates, or deletes a KA.++---++## Install & Run++```bash+pip install 'hyperextract[mcp]'++# start the server (stdio transport)+he-mcp+# equivalent:+python -m hyperextract.mcp_server+```++The server reads your LLM/embedder configuration from `~/.he/config.toml` — the same config the CLI uses, so run `he config init ...` first (see [Configuration](cli/configuration.md)).++---++## Connect it to an MCP client++Point your MCP client at the `he-mcp` command. For a Claude Desktop–style config:++```json+{+ "mcpServers": {+ "hyper-extract": {+ "command": "he-mcp"+ }+ }+}+```++---++## Tools++| Tool | Description | Needs index | Needs LLM |+|------|-------------|:-----------:|:---------:|+| `list_templates` | List available extraction templates | — | — |+| `info` | Stats for a KA (template, counts, index status) | — | — |+| `search` | Semantic retrieval over a KA | ✓ | embedder |+| `ask` | RAG question-answering over a KA | ✓ | ✓ |+| `export_obsidian` | Export a KA to an Obsidian vault | — | embedder |++All tools take a `ka_path` (a directory created by `he parse`). `search`/`ask` require an index — build it with [`he build-index`](cli/commands/build-index.md).++> `export_obsidian` requires the Obsidian export feature (see [`he export obsidian`](cli/commands/export.md)). If it is unavailable, the tool returns an explanatory message instead of failing.++---++## Example session++```text+list_templates() → [{name: "general/biography_graph", ...}, ...]+info(ka_path="./tesla_kb") → {nodes: 48, edges: 70, index_built: true, ...}+search(ka_path="./tesla_kb",+ query="War of Currents") → {nodes: [...], edges: [...]}+ask(ka_path="./tesla_kb",+ question="Who were Tesla's rivals?") → "Thomas Edison ..."+export_obsidian(ka_path="./tesla_kb",+ output="./vault") → "Exported 49 notes to ./vault"+```diff --git a/docs/zh/mcp.md b/docs/zh/mcp.mdnew file mode 100644index 0000000..ea9157f--- /dev/null+++ b/docs/zh/mcp.md@@ -0,0 +1,67 @@+# MCP 服务器++Hyper-Extract 提供了一个 [MCP](https://modelcontextprotocol.io) 服务器,让支持 MCP 的助手(Claude Desktop、IDE 智能体等)通过 Model Context Protocol **查询和导出你的知识摘要**。++它是**只读 + 导出**的——不会创建、修改或删除任何 KA。++---++## 安装与运行++```bash+pip install 'hyperextract[mcp]'++# 启动服务器(stdio 传输)+he-mcp+# 等价于:+python -m hyperextract.mcp_server+```++服务器从 `~/.he/config.toml` 读取 LLM/嵌入器配置——与 CLI 相同,因此请先运行 `he config init ...`(见 [配置](cli/configuration.md))。++---++## 接入 MCP 客户端++让你的 MCP 客户端指向 `he-mcp` 命令。以 Claude Desktop 风格的配置为例:++```json+{+ "mcpServers": {+ "hyper-extract": {+ "command": "he-mcp"+ }+ }+}+```++---++## 工具++| 工具 | 说明 | 需要索引 | 需要 LLM |+|------|-------------|:-----------:|:---------:|+| `list_templates` | 列出可用的提取模板 | — | — |+| `info` | KA 统计(模板、数量、索引状态) | — | — |+| `search` | KA 语义检索 | ✓ | 嵌入器 |+| `ask` | KA 上的 RAG 问答 | ✓ | ✓ |+| `export_obsidian` | 将 KA 导出为 Obsidian 知识库 | — | 嵌入器 |++所有工具都接受一个 `ka_path`(由 `he parse` 创建的目录)。`search`/`ask` 需要索引——用 [`he build-index`](cli/commands/build-index.md) 构建。++> `export_obsidian` 依赖 Obsidian 导出功能(见 [`he export obsidian`](cli/commands/export.md))。若不可用,该工具会返回说明信息而不会报错。++---++## 示例会话++```text+list_templates() → [{name: "general/biography_graph", ...}, ...]+info(ka_path="./tesla_kb") → {nodes: 48, edges: 70, index_built: true, ...}+search(ka_path="./tesla_kb",+ query="War of Currents") → {nodes: [...], edges: [...]}+ask(ka_path="./tesla_kb",+ question="Who were Tesla's rivals?") → "Thomas Edison ..."+export_obsidian(ka_path="./tesla_kb",+ output="./vault") → "Exported 49 notes to ./vault"+```diff --git a/hyperextract/mcp_server.py b/hyperextract/mcp_server.pynew file mode 100644index 0000000..89551db--- /dev/null+++ b/hyperextract/mcp_server.py@@ -0,0 +1,249 @@+"""MCP server exposing Hyper-Extract knowledge abstracts.++Lets MCP-capable assistants query and export existing Knowledge Abstracts over+the Model Context Protocol. Read + export only — it never creates, mutates, or+deletes a KA.++Tools:+ - list_templates : list available extraction templates+ - info : stats for a knowledge abstract (no LLM needed)+ - search : semantic retrieval over a KA (needs an index)+ - ask : RAG question-answering over a KA (needs an index)+ - export_obsidian : export a KA to an Obsidian vault++Run it (stdio transport):++ he-mcp+ # or+ python -m hyperextract.mcp_server++LLM/embedder come from ``~/.he/config.toml`` (same as the CLI). Requires the+optional extra: ``pip install 'hyperextract[mcp]'``.+"""++from __future__ import annotations++import json+from pathlib import Path+from typing import Any, Tuple++from langchain_core.embeddings import Embeddings+from langchain_core.language_models.chat_models import BaseChatModel++SERVER_NAME = "hyper-extract"+++# ---------------------------------------------------------------------------+# Internal helpers (kept module-level so they can be monkeypatched in tests)+# ---------------------------------------------------------------------------+++def _get_clients() -> Tuple[BaseChatModel, Embeddings]:+ """Return (llm, embedder) from the user's ~/.he/config.toml."""+ from hyperextract.utils.client import get_client++ return get_client()+++def _template_and_lang(path: Path) -> Tuple[str, str]:+ """Read the template name and language from a KA's metadata.json."""+ meta_path = path / "metadata.json"+ if not meta_path.exists():+ raise FileNotFoundError(f"No metadata.json in {path}")+ meta = json.loads(meta_path.read_text(encoding="utf-8"))+ template = meta.get("template")+ if not template:+ raise ValueError(f"No template recorded in {meta_path}")+ return template, meta.get("lang", "en")+++def _load_ka(ka_path: str):+ """Load a Knowledge Abstract from disk (needs LLM/embedder clients)."""+ from hyperextract.utils.template_engine import Template++ path = Path(ka_path)+ if not (path / "data.json").exists():+ raise FileNotFoundError(f"Not a knowledge abstract (no data.json): {ka_path}")+ template, lang = _template_and_lang(path)+ llm, emb = _get_clients()+ ka = Template.create(template, lang, llm_client=llm, embedder=emb)+ ka.load(path)+ return ka+++def _dump(obj: Any) -> str:+ return json.dumps(obj, ensure_ascii=False, indent=2, default=str)+++# ---------------------------------------------------------------------------+# Tool implementations (registered with FastMCP in build_server)+# ---------------------------------------------------------------------------+++def list_templates() -> str:+ """List the available knowledge-extraction templates.++ Returns a JSON array of {name, type, description}.+ """+ from hyperextract.utils.template_engine import Template++ templates = Template.list(include_methods=False)+ out = []+ for name, cfg in sorted(templates.items()):+ desc = getattr(cfg, "description", "") or ""+ if isinstance(desc, dict):+ desc = desc.get("en") or desc.get("zh") or ""+ out.append(+ {"name": name, "type": getattr(cfg, "type", None), "description": desc}+ )+ return _dump(out)+++def info(ka_path: str) -> str:+ """Show information and statistics for a knowledge abstract.++ Args:+ ka_path: Path to the knowledge abstract directory.++ Returns JSON with template, language, node/edge counts, and index status.+ Does not require LLM/embedder configuration.+ """+ path = Path(ka_path)+ data_file = path / "data.json"+ if not data_file.exists():+ return f"Not a knowledge abstract (no data.json): {ka_path}"++ data = json.loads(data_file.read_text(encoding="utf-8"))+ if isinstance(data, dict):+ nodes = len(data.get("nodes", data.get("entities", [])))+ edges = len(data.get("edges", data.get("relations", [])))+ elif isinstance(data, list):+ nodes, edges = len(data), 0+ else:+ nodes, edges = 0, 0++ meta: dict = {}+ meta_path = path / "metadata.json"+ if meta_path.exists():+ meta = json.loads(meta_path.read_text(encoding="utf-8"))++ index_dir = path / "index"+ index_built = index_dir.exists() and any(index_dir.iterdir())++ return _dump(+ {+ "path": str(path),+ "template": meta.get("template"),+ "lang": meta.get("lang"),+ "nodes": nodes,+ "edges": edges,+ "index_built": index_built,+ }+ )+++def search(ka_path: str, query: str, top_k: int = 5) -> str:+ """Semantic search over a knowledge abstract.++ Args:+ ka_path: Path to the knowledge abstract directory.+ query: Natural-language search query.+ top_k: Maximum number of results (for graphs: nodes and edges each).++ Returns matching nodes/edges as JSON. The KA must have an index+ (build it with `he build-index`).+ """+ ka = _load_ka(ka_path)+ try:+ results = ka.search(query, top_k=top_k)+ except ValueError as e:+ return f"Cannot search: {e}. Build the index first with `he build-index {ka_path}`."++ if isinstance(results, tuple):+ nodes, edges = results+ return _dump(+ {+ "nodes": [_model_to_dict(n) for n in nodes],+ "edges": [_model_to_dict(e) for e in edges],+ }+ )+ return _dump({"results": [_model_to_dict(r) for r in results]})+++def ask(ka_path: str, question: str, top_k: int = 5) -> str:+ """Answer a question using retrieval-augmented generation over a KA.++ Args:+ ka_path: Path to the knowledge abstract directory.+ question: The question to answer.+ top_k: Number of context items to retrieve.++ Returns the generated answer. The KA must have an index.+ """+ ka = _load_ka(ka_path)+ try:+ response = ka.chat(question, top_k=top_k)+ except ValueError as e:+ return f"Cannot answer: {e}. Build the index first with `he build-index {ka_path}`."+ return getattr(response, "content", str(response))+++def export_obsidian(+ ka_path: str,+ output: str,+ vault_name: str = "",+ overwrite: bool = False,+) -> str:+ """Export a knowledge abstract to an Obsidian vault.++ Args:+ ka_path: Path to the knowledge abstract directory.+ output: Destination vault directory.+ vault_name: Vault name for the index note (defaults to the output dir name).+ overwrite: Allow writing into an existing, non-empty directory.++ Returns a summary of how many notes were written.+ """+ ka = _load_ka(ka_path)+ if not hasattr(ka, "export_obsidian"):+ return (+ "Obsidian export is only supported for graph-type knowledge abstracts "+ "(graph, hypergraph, temporal/spatial graphs)."+ )+ name = vault_name or Path(output).name or "Knowledge Vault"+ try:+ vault = ka.export_obsidian(output, vault_name=name, overwrite=overwrite)+ except FileExistsError as e:+ return f"{e} Pass overwrite=true to write into it."+ count = len(list(Path(vault).glob("*.md")))+ return f"Exported {count} notes to {vault}"+++def _model_to_dict(item: Any) -> Any:+ if hasattr(item, "model_dump"):+ return item.model_dump()+ return item+++# ---------------------------------------------------------------------------+# Server wiring+# ---------------------------------------------------------------------------+++def build_server():+ """Build the FastMCP server with all tools registered."""+ from mcp.server.fastmcp import FastMCP++ server = FastMCP(SERVER_NAME)+ for fn in (list_templates, info, search, ask, export_obsidian):+ server.tool()(fn)+ return server+++def main() -> None:+ """Entry point: run the MCP server over stdio."""+ build_server().run()+++if __name__ == "__main__":+ main()diff --git a/mkdocs.yml b/mkdocs.ymlindex e1dfa42..589e583 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -81,6 +81,7 @@ plugins:- cli/commands/list.md- cli/commands/config.md- cli/configuration.md+ - MCP Server: mcp.md- Python SDK:- python/index.md- python/quickstart.md@@ -171,6 +172,7 @@ plugins:- zh/cli/commands/list.md- zh/cli/commands/config.md- zh/cli/configuration.md+ - MCP 服务器: zh/mcp.md- Python SDK:- zh/python/index.md- zh/python/quickstart.mddiff --git a/pyproject.toml b/pyproject.tomlindex 2cc55c6..0cb7ea9 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -58,8 +58,9 @@ dev = [[project.optional-dependencies]anthropic = ["langchain-anthropic>=0.3.0"]google = ["langchain-google-genai>=2.1.0"]+mcp = ["mcp>=1.2.0"]all = [- "hyperextract[anthropic,google]",+ "hyperextract[anthropic,google,mcp]",][project.urls]@@ -69,6 +70,7 @@ Issues = "https://github.com/yifanfeng97/hyper-extract/issues"[project.scripts]he = "hyperextract.cli:app"+he-mcp = "hyperextract.mcp_server:main"[build-system]requires = ["hatchling"]diff --git a/tests/test_mcp_server.py b/tests/test_mcp_server.pynew file mode 100644index 0000000..a8e8f8e--- /dev/null+++ b/tests/test_mcp_server.py@@ -0,0 +1,157 @@+"""Tests for the MCP server (hyperextract.mcp_server).++The tool functions are tested directly. The FastMCP wiring is smoke-tested and+skipped if the optional `mcp` package is not installed. The `mcp` package is not+required to import the module or test the tool logic.+"""++import json++import pytest+from pydantic import BaseModel, Field++from hyperextract import mcp_server+from hyperextract.types import AutoGraph+from tests.mocks import MockChatModel, MockEmbeddings+++class Entity(BaseModel):+ name: str+ type: str = "ENTITY"+ properties: dict = Field(default_factory=dict)+++class Relation(BaseModel):+ source: str+ target: str+ relation_type: str+++def _graph_with_index():+ g = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=MockChatModel(),+ embedder=MockEmbeddings(),+ )+ g._node_memory.add([Entity(name="Apple", type="ORG"), Entity(name="Steve Jobs")])+ g._edge_memory.add(+ [Relation(source="Apple", target="Steve Jobs", relation_type="founded_by")]+ )+ g.build_index()+ g.metadata["template"] = "general/base_graph"+ g.metadata["lang"] = "en"+ return g+++# ---------------------------------------------------------------------------+# list_templates+# ---------------------------------------------------------------------------+++def test_list_templates_returns_json():+ out = json.loads(mcp_server.list_templates())+ assert isinstance(out, list)+ assert len(out) > 0+ assert {"name", "type", "description"} <= set(out[0].keys())+++# ---------------------------------------------------------------------------+# info (no client needed — reads json directly)+# ---------------------------------------------------------------------------+++def test_info_reports_counts(tmp_path):+ g = _graph_with_index()+ ka = tmp_path / "ka"+ g.dump(ka)++ out = json.loads(mcp_server.info(str(ka)))+ assert out["nodes"] == 2+ assert out["edges"] == 1+ assert out["index_built"] is True+ assert out["template"] == "general/base_graph"+++def test_info_rejects_non_ka(tmp_path):+ out = mcp_server.info(str(tmp_path / "nope"))+ assert "Not a knowledge abstract" in out+++# ---------------------------------------------------------------------------+# search / ask / export — _load_ka monkeypatched to skip the Template reload+# ---------------------------------------------------------------------------+++def test_search_returns_nodes_and_edges(monkeypatch):+ g = _graph_with_index()+ monkeypatch.setattr(mcp_server, "_load_ka", lambda p: g)++ out = json.loads(mcp_server.search("x", "technology company", top_k=2))+ assert "nodes" in out and "edges" in out+ assert isinstance(out["nodes"], list)+++def test_search_without_index_is_handled(monkeypatch):+ g = AutoGraph(+ node_schema=Entity,+ edge_schema=Relation,+ node_key_extractor=lambda x: x.name,+ edge_key_extractor=lambda x: f"{x.source}-{x.relation_type}-{x.target}",+ nodes_in_edge_extractor=lambda x: (x.source, x.target),+ llm_client=MockChatModel(),+ embedder=MockEmbeddings(),+ )+ g._node_memory.add([Entity(name="Apple")])+ monkeypatch.setattr(mcp_server, "_load_ka", lambda p: g)++ out = mcp_server.search("x", "anything")+ assert "Cannot search" in out+ assert "build-index" in out+++class _StubKA:+ """Stub KA to test the MCP tool wiring independently of chat/export internals+ (those are covered by their own modules; this isolates the MCP layer)."""++ def chat(self, question, top_k=5):+ return type(+ "Resp", (), {"content": f"answer to: {question}", "additional_kwargs": {}}+ )()++ def export_obsidian(self, output, vault_name="", overwrite=False):+ from pathlib import Path++ out = Path(output)+ out.mkdir(parents=True, exist_ok=True)+ (out / "Note.md").write_text("x", encoding="utf-8")+ return out+++def test_ask_returns_answer(monkeypatch):+ monkeypatch.setattr(mcp_server, "_load_ka", lambda p: _StubKA())+ out = mcp_server.ask("x", "Who founded Apple?", top_k=2)+ assert out == "answer to: Who founded Apple?"+++def test_export_obsidian(monkeypatch, tmp_path):+ monkeypatch.setattr(mcp_server, "_load_ka", lambda p: _StubKA())+ vault = tmp_path / "vault"+ out = mcp_server.export_obsidian("x", str(vault), vault_name="KB", overwrite=True)+ assert "Exported 1 notes" in out+ assert (vault / "Note.md").exists()+++# ---------------------------------------------------------------------------+# FastMCP wiring (needs the optional `mcp` package)+# ---------------------------------------------------------------------------+++def test_build_server_registers_tools():+ pytest.importorskip("mcp")+ server = mcp_server.build_server()+ assert server is not None+ assert server.name == "hyper-extract"