Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.
feat: load-time identifiers validation; friendly local-model errors (0.10.5)
yifanfeng970dc002bb1029
diff --git a/README.md b/README.mdindex dfaddaf..c817e50 100644--- a/README.md+++ b/README.md@@ -354,7 +354,7 @@ identifiers:## 📰 What's New-**v0.10.4** — 👁️ Chunk extraction failures are now visible (`ka.extraction_failures`, optional `on_error="raise"`) · 🐛 Hypergraph incremental feed no longer crashes.+**v0.10.5** — 🛡️ Load-time template validation (undeclared identifier fields fail at create, not at merge) · 🧠 Native Gemini · 🔒 ontomem 0.6.1 safe merging.📰 **[Full release notes](https://yifanfeng97.github.io/Hyper-Extract/latest/news/)** · [All releases](https://github.com/yifanfeng97/hyper-extract/releases)diff --git a/README_ZH.md b/README_ZH.mdindex da5cd46..d8578a0 100644--- a/README_ZH.md+++ b/README_ZH.md@@ -354,7 +354,7 @@ identifiers:## 📰 最新动态-**v0.10.4** — 👁️ Chunk 抽取失败可见化(`ka.extraction_failures`,可选 `on_error="raise"`)· 🐛 超图增量喂入不再崩溃。+**v0.10.5** — 🛡️ 加载期模板校验(未声明字段在 create 阶段即报错)· 🧠 原生 Gemini · 🔒 ontomem 0.6.1 安全合并。📰 **[完整版本说明](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/news/)** · [全部 Releases](https://github.com/yifanfeng97/hyper-extract/releases)diff --git a/docs/en/news.md b/docs/en/news.mdindex 83f2108..d10002f 100644--- a/docs/en/news.md+++ b/docs/en/news.md@@ -4,6 +4,14 @@ Release notes and highlights. For a complete changelog, see the [GitHub releases---+## v0.10.5 — Load-Time Template Validation & Friendly Local-Model Errors++- **🛡️ Load-time identifiers validation** — a template whose `node_id`/`relation_id`/`relation_members` reference fields not declared in `output` now fails at `Template.create` with `relation_id references '{type}' but relations.fields does not declare 'type'`, instead of crashing at first merge with an unrelated `max()` error. *(production feedback; requires ontomem ≥ 0.6.1)*+- **🧠 Friendly local-model errors** — without a config file, `Template.create` no longer silently falls back to OpenAI defaults; the error lists the three ways forward (`he config init`, `-p vllm` local setup, explicit clients).+- **ontomem 0.6.1** — exact-rollback safety: `KeyExtractionError` on 100% key-extraction failure, aggregated `key_extraction_failed` warnings, and `remerge="mechanical"` for deterministic (LLM-free, seconds-fast) rollbacks on large shared keys. *(production feedback)*++---+## v0.10.4 — Visible Chunk Failures & Hypergraph Incremental Feed- **🐛 Hypergraph incremental feed fixed** — feeding a second document into an existing hypergraph KA crashed with `AttributeError: 'key_extractor'` (`_update_data_state` referenced a nonexistent attribute). Now uses `node_key_extractor`/`edge_key_extractor`.diff --git a/docs/zh/news.md b/docs/zh/news.mdindex 10d4b56..97eb8e7 100644--- a/docs/zh/news.md+++ b/docs/zh/news.md@@ -4,6 +4,14 @@---+## v0.10.5 — 加载期模板校验与本地模型友好报错++- **🛡️ 加载期 identifiers 校验** — 模板的 `node_id`/`relation_id`/`relation_members` 引用了 `output` 未声明字段时,`Template.create` 阶段即报 `relation_id references '{type}' but relations.fields does not declare 'type'`,而非首次合并时崩出无关的 `max()` 错误。*(生产反馈;需要 ontomem ≥ 0.6.1)*+- **🧠 本地模型友好报错** — 无配置文件时 `Template.create` 不再静默回退到 OpenAI 默认值,报错列出三条出路(`he config init`、`-p vllm` 本地方案、显式传客户端)。+- **ontomem 0.6.1** — 精确回滚安全:100% key 抽取失败时抛 `KeyExtractionError`、`key_extraction_failed` 警告聚合计数、`remerge="mechanical"` 大共享键确定性回滚(无 LLM、秒级)。*(生产反馈)*++---+## v0.10.4 — Chunk 失败可见化与超图增量喂入修复- **🐛 超图增量喂入修复** — 向已有超图 KA 喂第二个文档时崩溃(`_update_data_state` 引用了不存在的 `key_extractor` 属性)。现改用 `node_key_extractor`/`edge_key_extractor`。diff --git a/hyperextract/utils/client.py b/hyperextract/utils/client.pyindex 085ba1f..745575d 100644--- a/hyperextract/utils/client.py+++ b/hyperextract/utils/client.py@@ -600,9 +600,21 @@ def get_client(>>> # Or with custom config path>>> llm, emb = get_client("/path/to/config.toml")"""+ import os+from hyperextract.cli.config import ConfigManagerpath = Path(config_path) if config_path else DEFAULT_CONFIG_FILE+ if not path.exists() and not os.getenv("OPENAI_API_KEY"):+ raise FileNotFoundError(+ f"No Hyper-Extract config found at {path}. Choose one:\n"+ " 1. he config init -p openai -k YOUR_KEY\n"+ " 2. Local models: he config llm -p vllm -u http://localhost:8000/v1 "+ "-k dummy -m Qwen/Qwen3.5-9B (plus he config embedder -p vllm "+ "-u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3)\n"+ " 3. Pass clients explicitly: Template.create(..., "+ "llm_client=..., embedder=...)"+ )manager = ConfigManager(path)llm_config = manager.get_llm_config()diff --git a/hyperextract/utils/template_engine/factory.py b/hyperextract/utils/template_engine/factory.pyindex df20a03..ae6f617 100644--- a/hyperextract/utils/template_engine/factory.py+++ b/hyperextract/utils/template_engine/factory.py@@ -20,6 +20,7 @@ from .parsers import (parse_option,parse_output,)+from .parsers.identifiers import validate_identifiers_fieldsfrom .parsers.schemas.base import VALID_AUTOTYPESif TYPE_CHECKING:@@ -41,6 +42,17 @@ def _unknown_autotype_error(type_name: str) -> ValueError:return ValueError(f"Unknown template type {type_name!r}. Allowed types: {allowed}")+def _declared_output_fields(config) -> dict[str, set[str]]:+ """Declared field names per output section (entities/relations)."""+ declared: dict[str, set[str]] = {}+ output = config.output+ for section in ("entities", "relations"):+ fields = getattr(output, section, None)+ if fields is not None and getattr(fields, "fields", None) is not None:+ declared[section] = {f.name for f in fields.fields}+ return declared++class TemplateFactory:@classmethoddef create_method(@@ -183,6 +195,11 @@ class TemplateFactory:from hyperextract.types import AutoGraphentity_schema, relation_schema = parse_output(config.output, config.type)+ validate_identifiers_fields(+ config.identifiers,+ config.type,+ _declared_output_fields(config),+ )identifiers = parse_identifiers(config.identifiers, config.type)entity_key_extractor, relation_key_extractor, entities_in_relation_extractor = (identifiers@@ -223,6 +240,11 @@ class TemplateFactory:from hyperextract.types import AutoHypergraphentity_schema, relation_schema = parse_output(config.output, config.type)+ validate_identifiers_fields(+ config.identifiers,+ config.type,+ _declared_output_fields(config),+ )identifiers = parse_identifiers(config.identifiers, config.type)entity_key_extractor, relation_key_extractor, entities_in_relation_extractor = (identifiers@@ -262,6 +284,11 @@ class TemplateFactory:from hyperextract.types import AutoTemporalGraphentity_schema, relation_schema = parse_output(config.output, config.type)+ validate_identifiers_fields(+ config.identifiers,+ config.type,+ _declared_output_fields(config),+ )(entity_key_extractor,relation_key_extractor,@@ -306,6 +333,11 @@ class TemplateFactory:from hyperextract.types import AutoSpatialGraphentity_schema, relation_schema = parse_output(config.output, config.type)+ validate_identifiers_fields(+ config.identifiers,+ config.type,+ _declared_output_fields(config),+ )identifiers = parse_identifiers(config.identifiers, config.type)(entity_key_extractor,@@ -350,6 +382,11 @@ class TemplateFactory:from hyperextract.types import AutoSpatioTemporalGraphentity_schema, relation_schema = parse_output(config.output, config.type)+ validate_identifiers_fields(+ config.identifiers,+ config.type,+ _declared_output_fields(config),+ )identifiers = parse_identifiers(config.identifiers, config.type)(entity_key_extractor,diff --git a/hyperextract/utils/template_engine/parsers/identifiers.py b/hyperextract/utils/template_engine/parsers/identifiers.pyindex d6721b1..a401bbf 100644--- a/hyperextract/utils/template_engine/parsers/identifiers.py+++ b/hyperextract/utils/template_engine/parsers/identifiers.py@@ -82,6 +82,85 @@ def _members_extractor(return extractor+def _placeholders(field_or_template: str) -> list[str]:+ """Bracket placeholders of a template (simple field -> [itself])."""+ if "{" in field_or_template:+ return re.findall(r"\{(\w+)\}", field_or_template)+ return [field_or_template]+++def validate_identifiers_fields(+ identifiers: NaiveIdentifierSchema | GraphIdentifiersSchema,+ autotype: VALID_AUTOTYPES,+ declared_fields: dict[str, set[str]],+) -> None:+ """Load-time check: identifiers must reference declared output fields.++ A template whose ``relation_id`` cites ``{type}`` while+ ``relations.fields`` never declares ``type`` previously loaded fine and+ crashed at first merge with an unrelated ``max()`` error (every key+ extraction failed). This check fails at ``Template.create`` instead.++ Args:+ identifiers: identifiers config from YAML.+ autotype: auto type of the template.+ declared_fields: declared output field names, keyed by+ ``"entities"`` / ``"relations"``.++ Raises:+ ValueError: naming each referenced-but-undeclared field.+ """+ if autotype == "set":+ return+ if autotype not in (+ "graph",+ "hypergraph",+ "temporal_graph",+ "spatial_graph",+ "spatio_temporal_graph",+ ):+ return++ entity_fields = declared_fields.get("entities", set())+ relation_fields = declared_fields.get("relations", set())++ problems: list[str] = []+ for field in _placeholders(identifiers.entity_id):+ if field not in entity_fields:+ problems.append(+ f"node_id references '{{{field}}}' but "+ f"entities.fields does not declare '{field}'"+ )+ for field in _placeholders(identifiers.relation_id):+ if field not in relation_fields:+ problems.append(+ f"relation_id references '{{{field}}}' but "+ f"relations.fields does not declare '{field}'"+ )+ if isinstance(identifiers.relation_members, dict):+ for role, field in identifiers.relation_members.items():+ if field not in relation_fields:+ problems.append(+ f"relation_members['{role}'] references '{{{field}}}' but "+ f"relations.fields does not declare '{field}'"+ )+ for attr, section in (+ ("time_field", "entities"),+ ("location_field", "entities"),+ ):+ value = getattr(identifiers, attr, None)+ if value:+ for field in _placeholders(value):+ if field not in entity_fields:+ problems.append(+ f"{attr} references '{{{field}}}' but "+ f"entities.fields does not declare '{field}'"+ )++ if problems:+ raise ValueError("; ".join(problems))++def parse_identifiers(identifiers: NaiveIdentifierSchema | GraphIdentifiersSchema,autotype: VALID_AUTOTYPES,diff --git a/pyproject.toml b/pyproject.tomlindex f8f9493..237bcb7 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "hyperextract"-version = "0.10.4"+version = "0.10.5"description = "An intelligent, LLM-powered knowledge extraction and evolution framework with semantic search capabilities"readme = "README.md"requires-python = ">=3.11"@@ -33,7 +33,7 @@ dependencies = ["langchain-community>=0.4.1","langchain-openai>=1.1.7","structlog>=25.5.0",- "ontomem>=0.6.0",+ "ontomem>=0.6.1","ontosight>=0.1.8","python-dotenv>=1.2.1","semhash>=0.4.1",diff --git a/uv.lock b/uv.lockindex 047ddbe..45fc72f 100644--- a/uv.lock+++ b/uv.lock@@ -977,7 +977,7 @@ wheels = [[[package]]name = "hyperextract"-version = "0.10.4"+version = "0.10.5"source = { editable = "." }dependencies = [{ name = "faiss-cpu" },@@ -1037,7 +1037,7 @@ requires-dist = [{ name = "langchain-openai", specifier = ">=1.1.7" },{ name = "markitdown", extras = ["pdf", "docx", "pptx", "xlsx"], marker = "extra == 'ingest'", specifier = ">=0.1.4" },{ name = "mcp", marker = "extra == 'mcp'", specifier = ">=1.2.0,<3" },- { name = "ontomem", specifier = ">=0.6.0" },+ { name = "ontomem", specifier = ">=0.6.1" },{ name = "ontosight", specifier = ">=0.1.8" },{ name = "python-dotenv", specifier = ">=1.2.1" },{ name = "rich", specifier = ">=13.7.0" },@@ -2209,7 +2209,7 @@ wheels = [[[package]]name = "ontomem"-version = "0.6.0"+version = "0.6.1"source = { registry = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/" }dependencies = [{ name = "faiss-cpu" },@@ -2221,9 +2221,9 @@ dependencies = [{ name = "python-dotenv" },{ name = "structlog" },]-sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/b1/f8/b730a2070cad15d1ee0947d5f3e5d1e5d0780903d1668f51b93a88439867/ontomem-0.6.0.tar.gz", hash = "sha256:3c25b0c2ef050f477af5eecc3893a2071399007676ff52a8b909e4da6850b9c2", size = 2550918, upload-time = "2026-09-04T11:35:21.421Z" }+sdist = { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/ae/36/9ebddb5d165ba9fee6e7e54b98010c84fbbd75604f73b3a8143402f1f507/ontomem-0.6.1.tar.gz", hash = "sha256:5d5d96c5e392a52f76952cad84cd3ed45b2bd014aac07e775652f0f96d0a18ae", size = 2554899, upload-time = "2026-09-29T13:23:33.99Z" }wheels = [- { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/58/47/8c1074a571882b0380bd089f633b0e7d5e4cdeefd402791b9e62afe811ce/ontomem-0.6.0-py3-none-any.whl", hash = "sha256:6733e8780179b59b282bfabe317f63deeaf8853a963cf546c4722db8e4177a25", size = 47563, upload-time = "2026-09-04T11:35:19.508Z" },+ { url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/packages/c4/08/d9a4995b5225a444960804bcdfd90d68e11dd827bf6cfdf9b5d8d3203c38/ontomem-0.6.1-py3-none-any.whl", hash = "sha256:680493c62d495e2ce6d18c61c30529fa086bb849a2fa61fe3290aab0366e53d4", size = 49557, upload-time = "2026-09-29T13:23:32.268Z" },][[package]]