Skip to content

Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.

feat: load-time identifiers validation; friendly local-model errors (0.10.5)

yifanfeng970dc002bb1029
diff --git a/README.md b/README.md
index dfaddaf..c817e50 100644
--- a/README.md
+++ b/README.md
@@ -354,7 +354,7 @@ identifiers:
## 📰 What's New
-**v0.10.4** — 👁️ Chunk extraction failures are now visible (`ka.extraction_failures`, optional `on_error="raise"`) · 🐛 Hypergraph incremental feed no longer crashes.
+**v0.10.5** — 🛡️ Load-time template validation (undeclared identifier fields fail at create, not at merge) · 🧠 Native Gemini · 🔒 ontomem 0.6.1 safe merging.
📰 **[Full release notes](https://yifanfeng97.github.io/Hyper-Extract/latest/news/)** · [All releases](https://github.com/yifanfeng97/hyper-extract/releases)
diff --git a/README_ZH.md b/README_ZH.md
index da5cd46..d8578a0 100644
--- a/README_ZH.md
+++ b/README_ZH.md
@@ -354,7 +354,7 @@ identifiers:
## 📰 最新动态
-**v0.10.4** — 👁️ Chunk 抽取失败可见化(`ka.extraction_failures`,可选 `on_error="raise"`)· 🐛 超图增量喂入不再崩溃。
+**v0.10.5** — 🛡️ 加载期模板校验(未声明字段在 create 阶段即报错)· 🧠 原生 Gemini · 🔒 ontomem 0.6.1 安全合并。
📰 **[完整版本说明](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/news/)** · [全部 Releases](https://github.com/yifanfeng97/hyper-extract/releases)
diff --git a/docs/en/news.md b/docs/en/news.md
index 83f2108..d10002f 100644
--- a/docs/en/news.md
+++ b/docs/en/news.md
@@ -4,6 +4,14 @@ Release notes and highlights. For a complete changelog, see the [GitHub releases
---
+## v0.10.5 — Load-Time Template Validation & Friendly Local-Model Errors
+
+- **🛡️ Load-time identifiers validation** — a template whose `node_id`/`relation_id`/`relation_members` reference fields not declared in `output` now fails at `Template.create` with `relation_id references '{type}' but relations.fields does not declare 'type'`, instead of crashing at first merge with an unrelated `max()` error. *(production feedback; requires ontomem ≥ 0.6.1)*
+- **🧠 Friendly local-model errors** — without a config file, `Template.create` no longer silently falls back to OpenAI defaults; the error lists the three ways forward (`he config init`, `-p vllm` local setup, explicit clients).
+- **ontomem 0.6.1** — exact-rollback safety: `KeyExtractionError` on 100% key-extraction failure, aggregated `key_extraction_failed` warnings, and `remerge="mechanical"` for deterministic (LLM-free, seconds-fast) rollbacks on large shared keys. *(production feedback)*
+
+---
+
## v0.10.4 — Visible Chunk Failures & Hypergraph Incremental Feed
- **🐛 Hypergraph incremental feed fixed** — feeding a second document into an existing hypergraph KA crashed with `AttributeError: 'key_extractor'` (`_update_data_state` referenced a nonexistent attribute). Now uses `node_key_extractor`/`edge_key_extractor`.
diff --git a/docs/zh/news.md b/docs/zh/news.md
index 10d4b56..97eb8e7 100644
--- a/docs/zh/news.md
+++ b/docs/zh/news.md
@@ -4,6 +4,14 @@
---
+## v0.10.5 — 加载期模板校验与本地模型友好报错
+
+- **🛡️ 加载期 identifiers 校验** — 模板的 `node_id`/`relation_id`/`relation_members` 引用了 `output` 未声明字段时,`Template.create` 阶段即报 `relation_id references '{type}' but relations.fields does not declare 'type'`,而非首次合并时崩出无关的 `max()` 错误。*(生产反馈;需要 ontomem ≥ 0.6.1)*
+- **🧠 本地模型友好报错** — 无配置文件时 `Template.create` 不再静默回退到 OpenAI 默认值,报错列出三条出路(`he config init`、`-p vllm` 本地方案、显式传客户端)。
+- **ontomem 0.6.1** — 精确回滚安全:100% key 抽取失败时抛 `KeyExtractionError`、`key_extraction_failed` 警告聚合计数、`remerge="mechanical"` 大共享键确定性回滚(无 LLM、秒级)。*(生产反馈)*
+
+---
+
## v0.10.4 — Chunk 失败可见化与超图增量喂入修复
- **🐛 超图增量喂入修复** — 向已有超图 KA 喂第二个文档时崩溃(`_update_data_state` 引用了不存在的 `key_extractor` 属性)。现改用 `node_key_extractor`/`edge_key_extractor`。
diff --git a/hyperextract/utils/client.py b/hyperextract/utils/client.py
index 085ba1f..745575d 100644
--- a/hyperextract/utils/client.py
+++ b/hyperextract/utils/client.py
@@ -600,9 +600,21 @@ def get_client(
>>> # Or with custom config path
>>> llm, emb = get_client("/path/to/config.toml")
"""
+ import os
+
from hyperextract.cli.config import ConfigManager
path = Path(config_path) if config_path else DEFAULT_CONFIG_FILE
+ if not path.exists() and not os.getenv("OPENAI_API_KEY"):
+ raise FileNotFoundError(
+ f"No Hyper-Extract config found at {path}. Choose one:\n"
+ " 1. he config init -p openai -k YOUR_KEY\n"
+ " 2. Local models: he config llm -p vllm -u http://localhost:8000/v1 "
+ "-k dummy -m Qwen/Qwen3.5-9B (plus he config embedder -p vllm "
+ "-u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3)\n"
+ " 3. Pass clients explicitly: Template.create(..., "
+ "llm_client=..., embedder=...)"
+ )
manager = ConfigManager(path)
llm_config = manager.get_llm_config()
diff --git a/hyperextract/utils/template_engine/factory.py b/hyperextract/utils/template_engine/factory.py
index df20a03..ae6f617 100644
--- a/hyperextract/utils/template_engine/factory.py
+++ b/hyperextract/utils/template_engine/factory.py
@@ -20,6 +20,7 @@ from .parsers import (
parse_option,
parse_output,
)
+from .parsers.identifiers import validate_identifiers_fields
from .parsers.schemas.base import VALID_AUTOTYPES
if TYPE_CHECKING:
@@ -41,6 +42,17 @@ def _unknown_autotype_error(type_name: str) -> ValueError:
return ValueError(f"Unknown template type {type_name!r}. Allowed types: {allowed}")
+def _declared_output_fields(config) -> dict[str, set[str]]:
+ """Declared field names per output section (entities/relations)."""
+ declared: dict[str, set[str]] = {}
+ output = config.output
+ for section in ("entities", "relations"):
+ fields = getattr(output, section, None)
+ if fields is not None and getattr(fields, "fields", None) is not None:
+ declared[section] = {f.name for f in fields.fields}
+ return declared
+
+
class TemplateFactory:
@classmethod
def create_method(
@@ -183,6 +195,11 @@ class TemplateFactory:
from hyperextract.types import AutoGraph
entity_schema, relation_schema = parse_output(config.output, config.type)
+ validate_identifiers_fields(
+ config.identifiers,
+ config.type,
+ _declared_output_fields(config),
+ )
identifiers = parse_identifiers(config.identifiers, config.type)
entity_key_extractor, relation_key_extractor, entities_in_relation_extractor = (
identifiers
@@ -223,6 +240,11 @@ class TemplateFactory:
from hyperextract.types import AutoHypergraph
entity_schema, relation_schema = parse_output(config.output, config.type)
+ validate_identifiers_fields(
+ config.identifiers,
+ config.type,
+ _declared_output_fields(config),
+ )
identifiers = parse_identifiers(config.identifiers, config.type)
entity_key_extractor, relation_key_extractor, entities_in_relation_extractor = (
identifiers
@@ -262,6 +284,11 @@ class TemplateFactory:
from hyperextract.types import AutoTemporalGraph
entity_schema, relation_schema = parse_output(config.output, config.type)
+ validate_identifiers_fields(
+ config.identifiers,
+ config.type,
+ _declared_output_fields(config),
+ )
(
entity_key_extractor,
relation_key_extractor,
@@ -306,6 +333,11 @@ class TemplateFactory:
from hyperextract.types import AutoSpatialGraph
entity_schema, relation_schema = parse_output(config.output, config.type)
+ validate_identifiers_fields(
+ config.identifiers,
+ config.type,
+ _declared_output_fields(config),
+ )
identifiers = parse_identifiers(config.identifiers, config.type)
(
entity_key_extractor,
@@ -350,6 +382,11 @@ class TemplateFactory:
from hyperextract.types import AutoSpatioTemporalGraph
entity_schema, relation_schema = parse_output(config.output, config.type)
+ validate_identifiers_fields(
+ config.identifiers,
+ config.type,
+ _declared_output_fields(config),
+ )
identifiers = parse_identifiers(config.identifiers, config.type)
(
entity_key_extractor,
diff --git a/hyperextract/utils/template_engine/parsers/identifiers.py b/hyperextract/utils/template_engine/parsers/identifiers.py
index d6721b1..a401bbf 100644
--- a/hyperextract/utils/template_engine/parsers/identifiers.py
+++ b/hyperextract/utils/template_engine/parsers/identifiers.py
@@ -82,6 +82,85 @@ def _members_extractor(
return extractor
+def _placeholders(field_or_template: str) -> list[str]:
+ """Bracket placeholders of a template (simple field -> [itself])."""
+ if "{" in field_or_template:
+ return re.findall(r"\{(\w+)\}", field_or_template)
+ return [field_or_template]
+
+
+def validate_identifiers_fields(
+ identifiers: NaiveIdentifierSchema | GraphIdentifiersSchema,
+ autotype: VALID_AUTOTYPES,
+ declared_fields: dict[str, set[str]],
+) -> None:
+ """Load-time check: identifiers must reference declared output fields.
+
+ A template whose ``relation_id`` cites ``{type}`` while
+ ``relations.fields`` never declares ``type`` previously loaded fine and
+ crashed at first merge with an unrelated ``max()`` error (every key
+ extraction failed). This check fails at ``Template.create`` instead.
+
+ Args:
+ identifiers: identifiers config from YAML.
+ autotype: auto type of the template.
+ declared_fields: declared output field names, keyed by
+ ``"entities"`` / ``"relations"``.
+
+ Raises:
+ ValueError: naming each referenced-but-undeclared field.
+ """
+ if autotype == "set":
+ return
+ if autotype not in (
+ "graph",
+ "hypergraph",
+ "temporal_graph",
+ "spatial_graph",
+ "spatio_temporal_graph",
+ ):
+ return
+
+ entity_fields = declared_fields.get("entities", set())
+ relation_fields = declared_fields.get("relations", set())
+
+ problems: list[str] = []
+ for field in _placeholders(identifiers.entity_id):
+ if field not in entity_fields:
+ problems.append(
+ f"node_id references '{{{field}}}' but "
+ f"entities.fields does not declare '{field}'"
+ )
+ for field in _placeholders(identifiers.relation_id):
+ if field not in relation_fields:
+ problems.append(
+ f"relation_id references '{{{field}}}' but "
+ f"relations.fields does not declare '{field}'"
+ )
+ if isinstance(identifiers.relation_members, dict):
+ for role, field in identifiers.relation_members.items():
+ if field not in relation_fields:
+ problems.append(
+ f"relation_members['{role}'] references '{{{field}}}' but "
+ f"relations.fields does not declare '{field}'"
+ )
+ for attr, section in (
+ ("time_field", "entities"),
+ ("location_field", "entities"),
+ ):
+ value = getattr(identifiers, attr, None)
+ if value:
+ for field in _placeholders(value):
+ if field not in entity_fields:
+ problems.append(
+ f"{attr} references '{{{field}}}' but "
+ f"entities.fields does not declare '{field}'"
+ )
+
+ if problems:
+ raise ValueError("; ".join(problems))
+
+
def parse_identifiers(
identifiers: NaiveIdentifierSchema | GraphIdentifiersSchema,
autotype: VALID_AUTOTYPES,
diff --git a/pyproject.toml b/pyproject.toml
index f8f9493..237bcb7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[project]
name = "hyperextract"
-version = "0.10.4"
+version = "0.10.5"
description = "An intelligent, LLM-powered knowledge extraction and evolution framework with semantic search capabilities"
readme = "README.md"
requires-python = ">=3.11"
@@ -33,7 +33,7 @@ dependencies = [
"langchain-community>=0.4.1",
"langchain-openai>=1.1.7",
"structlog>=25.5.0",
- "ontomem>=0.6.0",
+ "ontomem>=0.6.1",
"ontosight>=0.1.8",
"python-dotenv>=1.2.1",
"semhash>=0.4.1",
diff --git a/uv.lock b/uv.lock
index 047ddbe..45fc72f 100644
--- a/uv.lock
+++ b/uv.lock
@@ -977,7 +977,7 @@ wheels = [
[[package]]
name = "hyperextract"
-version = "0.10.4"
+version = "0.10.5"
source = { editable = "." }
dependencies = [
{ name = "faiss-cpu" },
@@ -1037,7 +1037,7 @@ requires-dist = [
{ name = "langchain-openai", specifier = ">=1.1.7" },
{ name = "markitdown", extras = ["pdf", "docx", "pptx", "xlsx"], marker = "extra == 'ingest'", specifier = ">=0.1.4" },
{ name = "mcp", marker = "extra == 'mcp'", specifier = ">=1.2.0,<3" },
- { name = "ontomem", specifier = ">=0.6.0" },
+ { name = "ontomem", specifier = ">=0.6.1" },
{ name = "ontosight", specifier = ">=0.1.8" },
{ name = "python-dotenv", specifier = ">=1.2.1" },
{ name = "rich", specifier = ">=13.7.0" },
@@ -2209,7 +2209,7 @@ wheels = [
[[package]]
name = "ontomem"
-version = "0.6.0"
+version = "0.6.1"
source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }
dependencies = [
{ name = "faiss-cpu" },
@@ -2221,9 +2221,9 @@ dependencies = [
{ name = "python-dotenv" },
{ name = "structlog" },
]
-sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b1/f8/b730a2070cad15d1ee0947d5f3e5d1e5d0780903d1668f51b93a88439867/ontomem-0.6.0.tar.gz", hash = "sha256:3c25b0c2ef050f477af5eecc3893a2071399007676ff52a8b909e4da6850b9c2", size = 2550918, upload-time = "2026-09-04T11:35:21.421Z" }
+sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ae/36/9ebddb5d165ba9fee6e7e54b98010c84fbbd75604f73b3a8143402f1f507/ontomem-0.6.1.tar.gz", hash = "sha256:5d5d96c5e392a52f76952cad84cd3ed45b2bd014aac07e775652f0f96d0a18ae", size = 2554899, upload-time = "2026-09-29T13:23:33.99Z" }
wheels = [
- { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/58/47/8c1074a571882b0380bd089f633b0e7d5e4cdeefd402791b9e62afe811ce/ontomem-0.6.0-py3-none-any.whl", hash = "sha256:6733e8780179b59b282bfabe317f63deeaf8853a963cf546c4722db8e4177a25", size = 47563, upload-time = "2026-09-04T11:35:19.508Z" },
+ { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c4/08/d9a4995b5225a444960804bcdfd90d68e11dd827bf6cfdf9b5d8d3203c38/ontomem-0.6.1-py3-none-any.whl", hash = "sha256:680493c62d495e2ce6d18c61c30529fa086bb849a2fa61fe3290aab0366e53d4", size = 49557, upload-time = "2026-09-29T13:23:32.268Z" },
]
[[package]]