Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: add JSON-LD export for pairwise edges and hyperedges
Frankie-Xu5775471f6453
diff --git a/docs/en/cli/commands/export.md b/docs/en/cli/commands/export.mdindex eae88f8..950c7ba 100644--- a/docs/en/cli/commands/export.md+++ b/docs/en/cli/commands/export.md@@ -2,7 +2,7 @@Export a knowledge abstract to an [Obsidian](https://obsidian.md) vault — a folder of Markdown notes linked by `[[wikilinks]]`.-`export` is a command group. Formats: `obsidian`, `graphml`, `csv`.+`export` is a command group. Formats: `obsidian`, `graphml`, `jsonld`, `csv`.---@@ -165,6 +165,34 @@ he export graphml ./tesla_kb/ -o ./tesla.graphml---+## he export jsonld++Export a knowledge graph to JSON-LD (stdlib `json`, no RDFLib). Binary edges are `@type: Edge` with `source` / `target`. Edges with three or more endpoints are `@type: Hyperedge` with an `endpoint` list in extractor order (never sorted). 0/1-endpoint or missing-endpoint edges are skipped with a warning, same as GraphML.++Existing non-empty output files require `--force` / `-f`.++### Synopsis++```bash+he export jsonld KA_PATH -o FILE.jsonld [--force]+```++### Options++| Option | Alias | Default | Description |+|--------|-------|---------|-------------|+| `--output` | `-o` | *(required)* | Output JSON-LD file |+| `--force` | `-f` | off | Overwrite an existing non-empty file |++### Examples++```bash+he export jsonld ./tesla_kb/ -o ./tesla.jsonld+he export jsonld ./tesla_kb/ -o ./tesla.jsonld --force+```++---+## he export csvExport nodes and edges as CSV tables for spreadsheets and graph tools that ingest edge lists.@@ -215,12 +243,12 @@ he export csv ./tesla_kb/ -o ./tesla_csv/ --force## Python API-The same capability is available on graph Auto-Types for Obsidian, and as standalone functions for GraphML / CSV (they are not methods on AutoType):+The same capability is available on graph Auto-Types for Obsidian, and as standalone functions for GraphML / CSV / JSON-LD (they are not methods on AutoType):```pythonka.export_obsidian("./tesla_vault/", vault_name="Tesla KB", overwrite=True)-from hyperextract.utils.exporters import export_to_graphml, export_to_csv+from hyperextract.utils.exporters import export_to_graphml, export_to_csv, export_to_jsonldexport_to_graphml(ka.nodes,@@ -238,6 +266,14 @@ export_to_csv(folder_path="./tesla_csv/",overwrite=True,)++export_to_jsonld(+ ka.nodes,+ ka.edges,+ node_id_extractor=ka.node_key_extractor,+ incident_nodes_extractor=ka.nodes_in_edge_extractor,+ file_path="./tesla.jsonld",+)```---diff --git a/docs/en/cli/index.md b/docs/en/cli/index.mdindex 7b44d52..ade4009 100644--- a/docs/en/cli/index.md+++ b/docs/en/cli/index.md@@ -34,6 +34,7 @@ he --version| `he show` | Visualize knowledge graph | — || `he export obsidian` | Export to an Obsidian vault | `-o` output, `--name`, `-f` force || `he export graphml` | Export a pairwise graph to GraphML | `-o` output file |+| `he export jsonld` | Export pairwise edges and N-ary hyperedges to JSON-LD | `-o` output file, `-f` force || `he export csv` | Export nodes/edges as CSV tables | `-o` directory, `-f` force || `he search` | Semantic search in knowledge abstract | `-n` top-k results, `--source`, `--tag` || `he talk` | Chat with knowledge abstract | `-i` interactive, `-q` query |@@ -173,6 +174,7 @@ he show ./output/- **[`he info`](commands/info.md)** — View knowledge abstract statistics- **[`he export obsidian`](commands/export.md)** — Export to an Obsidian vault- **[`he export graphml`](commands/export.md#he-export-graphml)** — Export a pairwise graph to GraphML+- **[`he export jsonld`](commands/export.md#he-export-jsonld)** — Export pairwise edges and N-ary hyperedges to JSON-LD- **[`he export csv`](commands/export.md#he-export-csv)** — Export nodes/edges as CSV tables### Managementdiff --git a/docs/zh/cli/commands/export.md b/docs/zh/cli/commands/export.mdindex 5ad6c54..98d1e35 100644--- a/docs/zh/cli/commands/export.md+++ b/docs/zh/cli/commands/export.md@@ -2,7 +2,7 @@将知识摘要导出为 [Obsidian](https://obsidian.md) 知识库——一个由 `[[双向链接]]` 关联的 Markdown 笔记文件夹。-`export` 是一个命令组。目前支持的格式:`obsidian`、`graphml`、`csv`。+`export` 是一个命令组。目前支持的格式:`obsidian`、`graphml`、`jsonld`、`csv`。---@@ -165,6 +165,34 @@ he export graphml ./tesla_kb/ -o ./tesla.graphml---+## he export jsonld++将知识图谱导出为 JSON-LD(标准库 `json`,不引入 RDFLib)。二元边为 `@type: Edge`,带 `source` / `target`。三个及以上端点的边为 `@type: Hyperedge`,`endpoint` 列表保持抽取器顺序(不排序)。0/1 个端点或缺失端点的边会跳过并记 warning,与 GraphML 一致。++目标文件已存在且非空时需要 `--force` / `-f`。++### 用法++```bash+he export jsonld KA_PATH -o FILE.jsonld [--force]+```++### 选项++| 选项 | 别名 | 默认值 | 说明 |+|--------|-------|---------|-------------|+| `--output` | `-o` | *(必填)* | 输出 JSON-LD 文件 |+| `--force` | `-f` | 关闭 | 覆盖已存在的非空文件 |++### 示例++```bash+he export jsonld ./tesla_kb/ -o ./tesla.jsonld+he export jsonld ./tesla_kb/ -o ./tesla.jsonld --force+```++---+## he export csv将节点和边导出为 CSV 表,便于电子表格以及读取边列表的图谱工具使用。@@ -215,12 +243,12 @@ he export csv ./tesla_kb/ -o ./tesla_csv/ --force## Python API-Obsidian 导出是图谱 Auto-Type 上的方法;GraphML / CSV 是独立纯函数(不会加到 AutoType 上):+Obsidian 导出是图谱 Auto-Type 上的方法;GraphML / CSV / JSON-LD 是独立纯函数(不会加到 AutoType 上):```pythonka.export_obsidian("./tesla_vault/", vault_name="Tesla KB", overwrite=True)-from hyperextract.utils.exporters import export_to_graphml, export_to_csv+from hyperextract.utils.exporters import export_to_graphml, export_to_csv, export_to_jsonldexport_to_graphml(ka.nodes,@@ -238,6 +266,14 @@ export_to_csv(folder_path="./tesla_csv/",overwrite=True,)++export_to_jsonld(+ ka.nodes,+ ka.edges,+ node_id_extractor=ka.node_key_extractor,+ incident_nodes_extractor=ka.nodes_in_edge_extractor,+ file_path="./tesla.jsonld",+)```---diff --git a/docs/zh/cli/index.md b/docs/zh/cli/index.mdindex e8b2b64..0c25e6f 100644--- a/docs/zh/cli/index.md+++ b/docs/zh/cli/index.md@@ -34,6 +34,7 @@ he --version| `he show` | 可视化知识图谱 | — || `he export obsidian` | 导出为 Obsidian 知识库 | `-o` 输出, `--name`, `-f` 强制 || `he export graphml` | 将二元图导出为 GraphML | `-o` 输出文件 |+| `he export jsonld` | 将二元边与 N 元超边导出为 JSON-LD | `-o` 输出文件, `-f` 强制 || `he export csv` | 将节点/边导出为 CSV 表 | `-o` 目录, `-f` 强制 || `he search` | 知识库语义搜索 | `-n` top-k 结果数, `--source`, `--tag` || `he talk` | 与知识库对话 | `-i` 交互模式, `-q` 查询 |@@ -173,6 +174,7 @@ he show ./output/- **[`he info`](commands/info.md)** — 查看知识库统计信息- **[`he export obsidian`](commands/export.md)** — 导出为 Obsidian 知识库- **[`he export graphml`](commands/export.md#he-export-graphml)** — 将二元图导出为 GraphML+- **[`he export jsonld`](commands/export.md#he-export-jsonld)** — 将二元边与 N 元超边导出为 JSON-LD- **[`he export csv`](commands/export.md#he-export-csv)** — 将节点/边导出为 CSV 表### 管理diff --git a/hyperextract/cli/cli.py b/hyperextract/cli/cli.pyindex 4f6d33d..1b82895 100644--- a/hyperextract/cli/cli.py+++ b/hyperextract/cli/cli.py@@ -574,8 +574,8 @@ def _load_graph_ka_for_export(ka_path: str):if not hasattr(ka, "export_obsidian"):console.print(- "[red]Error:[/red] GraphML/CSV export is only supported for graph-type "- "Knowledge Abstracts (graph, hypergraph, temporal/spatial graphs)."+ "[red]Error:[/red] GraphML/CSV/JSON-LD export is only supported for "+ "graph-type Knowledge Abstracts (graph, hypergraph, temporal/spatial graphs).")raise typer.Exit(1)@@ -635,6 +635,60 @@ def export_graphml_cmd(console.print("[dim]Open the file in Gephi, yEd, or another GraphML tool.[/dim]")+@export_app.command(name="jsonld")+def export_jsonld_cmd(+ ka_path: str = typer.Argument(..., help="Knowledge Abstract directory"),+ output: str = typer.Option(..., "--output", "-o", help="Output JSON-LD file"),+ force: bool = typer.Option(+ False, "--force", "-f", help="Overwrite an existing JSON-LD file"+ ),+):+ """Export a knowledge graph to JSON-LD.++ Binary edges are `@type: Edge` with source/target. Edges with three+ or more endpoints are `@type: Hyperedge` with an endpoint list.+ """+ from hyperextract.utils.exporters import export_to_jsonld++ logger.info("command=export-jsonld ka_path=%s output=%s", ka_path, output)++ output_path = Path(output)+ existing_nonempty = (+ output_path.exists()+ and output_path.is_file()+ and output_path.stat().st_size > 0+ )+ if existing_nonempty and not force:+ console.print(+ "[red]Error:[/red] Output file already exists. "+ "Use --force / -f to overwrite it."+ )+ raise typer.Exit(1)++ ka, _path, template = _load_graph_ka_for_export(ka_path)+ console.print(f"[blue]Knowledge Abstract:[/blue] {ka_path}")+ console.print(f"[blue]Template:[/blue] {template}")+ console.print(f"[blue]Output file:[/blue] {output}")+ console.print()++ with console.status("[bold blue]Exporting to JSON-LD..."):+ try:+ export_to_jsonld(+ ka.nodes,+ ka.edges,+ node_id_extractor=ka.node_key_extractor,+ incident_nodes_extractor=ka.nodes_in_edge_extractor,+ file_path=output_path,+ edge_id_extractor=getattr(ka, "edge_key_extractor", None),+ )+ except Exception as e:+ console.print(f"[red]Error during export:[/red] {e}")+ raise typer.Exit(1)++ console.print()+ console.print(f"[bold green]Success![/bold green] Wrote JSON-LD to {output_path}")++@export_app.command(name="csv")def export_csv_cmd(ka_path: str = typer.Argument(..., help="Knowledge Abstract directory"),diff --git a/hyperextract/utils/exporters/__init__.py b/hyperextract/utils/exporters/__init__.pyindex f157b45..cf0a7da 100644--- a/hyperextract/utils/exporters/__init__.py+++ b/hyperextract/utils/exporters/__init__.py@@ -1,4 +1,4 @@-"""Read-only graph exporters (GraphML, CSV).+"""Read-only graph exporters (GraphML, CSV, JSON-LD).These are pure functions over node/edge models, matching:func:`hyperextract.utils.obsidian.export_to_obsidian`. They are not methods@@ -20,10 +20,12 @@ Example::from .common import HYPEREDGE_MEMBER_SEPfrom .csv_export import export_to_csvfrom .graphml import GraphMLHypergraphError, export_to_graphml+from .jsonld import export_to_jsonld__all__ = ["HYPEREDGE_MEMBER_SEP","GraphMLHypergraphError","export_to_csv","export_to_graphml",+ "export_to_jsonld",]diff --git a/hyperextract/utils/exporters/jsonld.py b/hyperextract/utils/exporters/jsonld.pynew file mode 100644index 0000000..6644c01--- /dev/null+++ b/hyperextract/utils/exporters/jsonld.py@@ -0,0 +1,154 @@+"""JSON-LD exporter for pairwise edges and GraphML-style hyperedges.++Produces a JSON-LD 1.1 document (stdlib ``json``, no RDFLib). Binary edges+are ``@type: Edge`` with ``source`` / ``target``. Edges with three or more+endpoints are ``@type: Hyperedge`` with an ``endpoint`` list in+``incident_nodes_extractor`` order. Endpoints are never sorted.+"""++import json+from collections.abc import Callable, Sequence+from pathlib import Path+from typing import Any++from pydantic import BaseModel++from hyperextract.utils.logging import get_logger++from .common import incident_ids, resolve_nodes, scalar_fields++logger = get_logger(__name__)++# Inline context: only the keys the CLI / interop contract documents.+JSONLD_CONTEXT: dict[str, Any] = {+ "Node": "Node",+ "Edge": "Edge",+ "Hyperedge": "Hyperedge",+ "source": {"@type": "@id"},+ "target": {"@type": "@id"},+ "endpoint": {"@container": "@list"},+}++_RESERVED_KEYS = frozenset({"@id", "@type", "source", "target", "endpoint"})+++def export_to_jsonld(+ nodes: Sequence[BaseModel],+ edges: Sequence[BaseModel],+ *,+ node_id_extractor: Callable[[Any], str],+ incident_nodes_extractor: Callable[[Any], Sequence[str]],+ file_path: str | Path,+ edge_id_extractor: Callable[[Any], str] | None = None,+) -> Path:+ """Export a graph to a JSON-LD file.++ Args:+ nodes: Node/entity models to export.+ edges: Edge models to export (pairwise or N-ary).+ node_id_extractor: Maps a node to its unique key (matches edge endpoints).+ incident_nodes_extractor: Maps an edge to incident node keys in order.+ Two endpoints become ``Edge``; three or more become ``Hyperedge``.+ file_path: Destination ``.jsonld`` file.+ edge_id_extractor: Optional edge -> ``@id``. Defaults to ``e0``, ``e1``, ...++ Returns:+ The written file :class:`~pathlib.Path`.++ Raises:+ IsADirectoryError: If ``file_path`` exists and is a directory.+ """+ dest = Path(file_path)+ if dest.exists() and dest.is_dir():+ raise IsADirectoryError(+ f"Destination '{dest}' is a directory; pass a JSON-LD file path."+ )+ dest.parent.mkdir(parents=True, exist_ok=True)++ by_id = resolve_nodes(nodes, node_id_extractor)+ graph: list[dict[str, Any]] = []+ for node_id, node in by_id.items():+ item: dict[str, Any] = {"@id": node_id, "@type": "Node"}+ item.update(_public_fields(node))+ graph.append(item)++ skipped = 0+ for index, edge in enumerate(edges):+ members = incident_ids(edge, incident_nodes_extractor)+ if members is None:+ skipped += 1+ continue+ if len(members) <= 1:+ skipped += 1+ logger.warning(+ "export.jsonld: skipping edge with %d endpoint(s) members=%s",+ len(members),+ members,+ )+ continue+ missing = [member for member in members if member not in by_id]+ if missing:+ skipped += 1+ logger.warning(+ "export.jsonld: skipping edge with missing endpoint(s) %s",+ missing,+ )+ continue+ edge_id = _edge_id(edge, index, edge_id_extractor)+ fields = _public_fields(edge)+ if len(members) == 2:+ item = {+ "@id": edge_id,+ "@type": "Edge",+ "source": members[0],+ "target": members[1],+ }+ item.update(fields)+ graph.append(item)+ continue+ item = {+ "@id": edge_id,+ "@type": "Hyperedge",+ "endpoint": list(members),+ }+ item.update(fields)+ graph.append(item)++ document = {"@context": JSONLD_CONTEXT, "@graph": graph}+ dest.write_text(+ json.dumps(document, ensure_ascii=False, indent=2) + "\n",+ encoding="utf-8",+ )++ edge_count = sum(1 for item in graph if item.get("@type") == "Edge")+ hyper_count = sum(1 for item in graph if item.get("@type") == "Hyperedge")+ logger.info(+ "jsonld: exported nodes=%d edges=%d hyperedges=%d skipped_edges=%d path=%s",+ len(by_id),+ edge_count,+ hyper_count,+ skipped,+ dest,+ )+ return dest+++def _public_fields(model: BaseModel) -> dict[str, str | int | float | bool]:+ return {+ key: value+ for key, value in scalar_fields(model).items()+ if key not in _RESERVED_KEYS+ }+++def _edge_id(edge: Any, index: int, extractor: Callable[[Any], str] | None) -> str:+ if extractor is None:+ return f"e{index}"+ try:+ value = extractor(edge)+ except Exception as exc:+ logger.debug("export.jsonld: edge_id_extractor raised %s", exc)+ return f"e{index}"+ if value in (None, ""):+ return f"e{index}"+ return str(value)diff --git a/tests/utils/test_exporters.py b/tests/utils/test_exporters.pyindex 4e54e67..9ee3534 100644--- a/tests/utils/test_exporters.py+++ b/tests/utils/test_exporters.py@@ -19,6 +19,7 @@ from hyperextract.utils.exporters import (HYPEREDGE_MEMBER_SEP,export_to_csv,export_to_graphml,+ export_to_jsonld,)from hyperextract.utils.exporters.graphml import GRAPHML_NS@@ -206,6 +207,88 @@ class TestGraphMLPairwise:assert _edge_endpoints(graph, ns) == []+# ---------------------------------------------------------------------------+# JSON-LD — pairwise source/target, N-ary Hyperedge order+# ---------------------------------------------------------------------------+++def _load_jsonld(path):+ return json.loads(path.read_text(encoding="utf-8"))+++class TestJSONLDExport:+ def test_pairwise_has_source_target(self, tmp_path):+ nodes = [Entity(name="A"), Entity(name="B")]+ edges = [Relation(source="B", target="A", relation_type="leads_to")]+ path = export_to_jsonld(+ nodes,+ edges,+ node_id_extractor=lambda n: n.name,+ incident_nodes_extractor=lambda e: (e.source, e.target),+ file_path=tmp_path / "g.jsonld",+ )+ doc = _load_jsonld(path)+ assert "source" in doc["@context"]+ assert doc["@context"]["source"]["@type"] == "@id"+ assert doc["@context"]["endpoint"]["@container"] == "@list"+ edges_out = [item for item in doc["@graph"] if item.get("@type") == "Edge"]+ assert len(edges_out) == 1+ assert edges_out[0]["source"] == "B"+ assert edges_out[0]["target"] == "A"+ assert edges_out[0]["@type"] == "Edge"+ assert not any(item.get("@type") == "Hyperedge" for item in doc["@graph"])++ def test_nary_writes_hyperedge_in_extractor_order(self, tmp_path):+ nodes = [Entity(name="A"), Entity(name="B"), Entity(name="C")]+ edges = [Event(label="meeting", participants=["C", "A", "B"])]+ path = export_to_jsonld(+ nodes,+ edges,+ node_id_extractor=lambda n: n.name,+ incident_nodes_extractor=lambda e: tuple(e.participants),+ file_path=tmp_path / "g.jsonld",+ edge_id_extractor=lambda e: e.label,+ )+ doc = _load_jsonld(path)+ hypers = [item for item in doc["@graph"] if item.get("@type") == "Hyperedge"]+ assert len(hypers) == 1+ assert hypers[0]["@type"] == "Hyperedge"+ assert hypers[0]["endpoint"] == ["C", "A", "B"]+ assert hypers[0]["endpoint"] != sorted(hypers[0]["endpoint"])+ assert not any(item.get("@type") == "Edge" for item in doc["@graph"])++ def test_unary_and_missing_endpoints_are_skipped(self, tmp_path):+ nodes = [Entity(name="Apple")]+ edges = [+ Event(label="solo", participants=["Apple"]),+ Relation(source="Apple", target="Ghost", relation_type="x"),+ ]++ def _incident(edge):+ if isinstance(edge, Event):+ return tuple(edge.participants)+ return (edge.source, edge.target)++ path = export_to_jsonld(+ nodes,+ edges,+ node_id_extractor=lambda n: n.name,+ incident_nodes_extractor=_incident,+ file_path=tmp_path / "g.jsonld",+ )+ doc = _load_jsonld(path)+ typed = [+ item+ for item in doc["@graph"]+ if item.get("@type") in ("Edge", "Hyperedge")+ ]+ nodes_out = [+ item["@id"] for item in doc["@graph"] if item.get("@type") == "Node"+ ]+ assert typed == []+ assert nodes_out == ["Apple"]++# ---------------------------------------------------------------------------# CSV — round-trip, quoting, direction, dangling, hypergraph# ---------------------------------------------------------------------------@@ -484,3 +567,85 @@ class TestCLIExport:)assert result.exit_code == 1assert "graph" in result.output.lower()++ def test_jsonld_writes_pairwise_and_hyperedge(self, tmp_path):+ ka_dir = _ka_dir(tmp_path)+ fake = FakeGraphKA(+ [Entity(name="A"), Entity(name="B")],+ [Relation(source="B", target="A", relation_type="leads_to")],+ )+ out = tmp_path / "out.jsonld"+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch(+ "hyperextract.cli.cli.get_template_from_ka", return_value=("t", "en")+ ),+ patch("hyperextract.cli.cli.Template.create", return_value=fake),+ ):+ result = runner.invoke(+ app, ["export", "jsonld", str(ka_dir), "-o", str(out)]+ )+ assert result.exit_code == 0, result.output+ doc = _load_jsonld(out)+ edges_out = [item for item in doc["@graph"] if item.get("@type") == "Edge"]+ assert edges_out[0]["source"] == "B"+ assert edges_out[0]["target"] == "A"++ fake_h = FakeGraphKA(+ [Entity(name="A"), Entity(name="B"), Entity(name="C")],+ [Event(label="meeting", participants=["C", "A", "B"])],+ hypergraph=True,+ )+ out_h = tmp_path / "h.jsonld"+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch(+ "hyperextract.cli.cli.get_template_from_ka", return_value=("t", "en")+ ),+ patch("hyperextract.cli.cli.Template.create", return_value=fake_h),+ ):+ result = runner.invoke(+ app, ["export", "jsonld", str(ka_dir), "-o", str(out_h)]+ )+ assert result.exit_code == 0, result.output+ doc_h = _load_jsonld(out_h)+ hypers = [item for item in doc_h["@graph"] if item.get("@type") == "Hyperedge"]+ assert hypers[0]["endpoint"] == ["C", "A", "B"]++ def test_jsonld_requires_force_for_existing_file(self, tmp_path):+ ka_dir = _ka_dir(tmp_path)+ dest = tmp_path / "out.jsonld"+ dest.write_text("SENTINEL", encoding="utf-8")+ fake = FakeGraphKA([Entity(name="A")], [])+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch(+ "hyperextract.cli.cli.get_template_from_ka", return_value=("t", "en")+ ),+ patch("hyperextract.cli.cli.Template.create", return_value=fake),+ ):+ result = runner.invoke(+ app, ["export", "jsonld", str(ka_dir), "-o", str(dest)]+ )+ assert result.exit_code != 0+ assert "--force" in result.output or "-f" in result.output+ assert dest.read_text(encoding="utf-8") == "SENTINEL"++ def test_jsonld_force_overwrites_existing_file(self, tmp_path):+ ka_dir = _ka_dir(tmp_path)+ dest = tmp_path / "out.jsonld"+ dest.write_text("SENTINEL", encoding="utf-8")+ fake = FakeGraphKA([Entity(name="A")], [])+ with (+ patch("hyperextract.cli.cli.validate_config"),+ patch(+ "hyperextract.cli.cli.get_template_from_ka", return_value=("t", "en")+ ),+ patch("hyperextract.cli.cli.Template.create", return_value=fake),+ ):+ result = runner.invoke(+ app, ["export", "jsonld", str(ka_dir), "-o", str(dest), "--force"]+ )+ assert result.exit_code == 0, result.output+ doc = _load_jsonld(dest)+ assert any(item.get("@type") == "Node" for item in doc["@graph"])