Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

feat(validate): 0/1/2 exit-code policy + lift the validate Markdown renderer (#994)

Louis Choquel80eb8a4ab9e4
diff --git a/.vscode/settings.json b/.vscode/settings.json
index 4c3156e1d..87cb4fb30 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -1,6 +1,5 @@
{
"python.defaultInterpreterPath": "${workspaceFolder}/.venv/bin/python",
- "python.analysis.extraPaths": ["./tests"],
"[python]": {
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true,
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 689bab200..0857231cb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,20 @@
# Changelog
+## [Unreleased]
+
+### Added
+
+- **Structured wiring/concept/type validation errors** — bundle-validation failures that previously collapsed to a bare message-only `blueprint_validation` residual now emit a structured, categorized `validation_errors[]` item with identity locators. A **pipe-owned unresolved concept reference** (a pipe input/output) → `pipe_validation` / `unresolved_concept` (with the referencing `pipe_code`, the missing `concept_code`, and the `field_name`); a **concept-owned unresolved concept reference** (a concept's `refines` or a structure field's `concept_ref`) → `blueprint_validation` / `unresolved_concept` (with the owning `concept_code`); an **undefined pipe dependency** → `pipe_validation` / `unresolved_pipe_dependency` (with the referencing `pipe_code` and the missing dependency in the new `missing_pipe_code` locator); an **unknown pipe `type`** (a pydantic discriminated-union tag failure) → `blueprint_validation` / `unknown_pipe_type` (with the `pipe_code`). New `PipeValidationErrorType` values: `unresolved_concept`, `unresolved_pipe_dependency`, `unknown_pipe_type`. New `validation_errors[]` locator field: `missing_pipe_code`. `ConceptLibraryError` now extends `LibraryLoadingError` so it carries the per-reference structured items through the existing error cascade (its raised type and message are unchanged; sites that pass only a message behave as before). Machine consumers of `validation_errors[]` get `error_type` + locators for these cases instead of only a `message`.
+
+### Changed (pre-1.0 breaking)
+
+- **Validate exit-code policy (0 / 1 / 2)** — the `validate` surface (both the bare `pipelex validate {bundle,method,pipe}` group and `pipelex-agent validate`) now exits with three codes mirroring the hosted `/validate` 200-verdict-vs-non-2xx-no-verdict split: `0` = valid, `1` = a produced negative verdict (invalid bundle, or valid-but-not-runnable without `--allow-signatures`), `2` = no verdict (bad args, unresolvable target, setup/internal error). Previously every failure exited `1`. Notably, `pipelex validate bundle` **directory-resolution failures** (no `.mthds` found / multiple `.mthds` and no default `bundle.mthds` / a path that is neither a file nor a directory) now exit **2** instead of **1**. Both `1` and `2` remain non-zero, so consumers that only test zero-vs-non-zero are unaffected. The verdict remains in the structured `is_valid` field — machine consumers should read it rather than branch on the exit code.
+- **Validate Markdown renderer relocated** — `format_validate_markdown` moved from `pipelex.cli.agent_cli.commands.validate._output_helpers` to the public, non-CLI module `pipelex.pipeline.validation_render`, so other surfaces (e.g. `pipelex-api`'s `/validate` opt-in `rendered_markdown` extra) can import it without pulling in the CLI/Typer stack. Output is byte-identical; only the import path changed.
+
+### Fixed
+
+- **`pipe_code` dropped on categorized pipe blueprint errors** — the blueprint validation-error categorizer matched the wrong pydantic `loc` key (`pipes` instead of the actual blueprint field name `pipe`), so every categorized pipe blueprint item (e.g. `missing_input_variable`) silently lost its `pipe_code` locator. It now populates correctly.
+
## [v0.34.0] - 2026-06-17
### Highlights
diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.md
index eb9908392..35fb559b8 100644
--- a/docs/under-the-hood/error-model.md
+++ b/docs/under-the-hood/error-model.md
@@ -306,6 +306,20 @@ The agent CLI (`pipelex-agent`) emits a structured error to **stderr**, markdown
The human CLI (`pipelex`) renders a Rich error panel — red banner, structured fields, the `user_action` tip, doc/Discord links — through the shared `display_error_panel()` helper in `pipelex/cli/error_handlers.py`.
+#### Validate exit-code policy (0 / 1 / 2)
+
+The `validate` surface — both the bare `pipelex validate {bundle,method,pipe}` group and the agent CLI's `pipelex-agent validate` — exits with **three** codes that mirror the hosted `/validate` 200-verdict-vs-non-2xx-no-verdict split:
+
+| Exit | Class | Condition |
+|------|-------|-----------|
+| `0` | valid | `is_valid` — including valid-but-not-runnable **with** `--allow-signatures` |
+| `1` | negative verdict | a produced "no": an invalid bundle (`ValidateBundleError`), or valid-but-not-runnable **without** `--allow-signatures` (a strict signature breach) |
+| `2` | no verdict | the CLI could not produce a verdict — bad args, an unresolvable target (no `.mthds` in a directory, a missing file, an unknown/ambiguous pipe code), or a setup/internal error during validate |
+
+**The verdict lives in the structured `is_valid` field, not the exit code.** The exit code is a convenience signal for naive shell/CI/Makefile use (`set -e`, `cmd && next`, `if cmd; then`); machine consumers (hooks, the Codex hook, runners) MUST read `is_valid` (and `error_domain`) from the JSON for their block/warn decisions rather than branching on the exit code. Decoupling the verdict from the exit code is what keeps any future exit-code change non-breaking. The 1-vs-2 split is also additive for flat consumers: both stay non-zero, so anything that only tests zero-vs-non-zero is unaffected.
+
+Implementation: the agent CLI threads `exit_code` through `agent_error(...)` (`agent_output.py`, default 1); the validate commands pass `exit_code=2` at every no-verdict site and keep the default 1 on the `ValidateBundleError` arm and the signature gate. The bare CLI sets the code directly via `typer.Exit(...)` in `cli/commands/validate/*` and via the `exit_code` parameter on `handle_model_choice_error` / `handle_model_availability_error` in `cli/error_handlers.py`. Shared boot handlers (`make_pipelex_for_cli`'s gateway/inference/telemetry/model-deck-preset paths) stay exit 1 — they are shared across `run`/`build`/`validate` and out of the validate-policy scope.
+
### API
`pipelex` is a library — there is no API server in the package. Downstream HTTP repos consume the `ErrorReport`:
@@ -319,7 +333,7 @@ A downstream FastAPI exception handler calls `ErrorReport.http_status` and is a
**Inputs.** `to_error_report()` takes a live `PipelexError`. `recover_error_report()` takes any `BaseException` and walks its `__cause__` chain. `ErrorReport.from_dict()` takes a `to_dict()` payload — strictly, raising `ValidationError` on drift.
-**Outputs.** `to_error_report()` returns an `ErrorReport`; `to_dict()` returns a `None`-free `dict`. Side effects: telemetry events emitted on pipeline failure at Layer 3; the agent CLI writes to stderr and raises `typer.Exit(1)`.
+**Outputs.** `to_error_report()` returns an `ErrorReport`; `to_dict()` returns a `None`-free `dict`. Side effects: telemetry events emitted on pipeline failure at Layer 3; the agent CLI writes to stderr and raises `typer.Exit(...)` — code 1 by default, or the validate surface's 0/1/2 policy (see [Validate exit-code policy](#validate-exit-code-policy-0--1--2)).
---
diff --git a/pipelex/base_exceptions.py b/pipelex/base_exceptions.py
index 3e7a3f73b..bce4bc912 100644
--- a/pipelex/base_exceptions.py
+++ b/pipelex/base_exceptions.py
@@ -302,6 +302,7 @@ class ValidationErrorItem(BaseModel):
field_name: str | None = None
variable_names: list[str] | None = None
missing_concept_code: str | None = None
+ missing_pipe_code: str | None = None
declared_concepts: list[str] | None = None
diff --git a/pipelex/cli/agent_cli/CLAUDE.md b/pipelex/cli/agent_cli/CLAUDE.md
index 8039a2183..8fe7425b6 100644
--- a/pipelex/cli/agent_cli/CLAUDE.md
+++ b/pipelex/cli/agent_cli/CLAUDE.md
@@ -89,7 +89,7 @@ commands/
## Key Patterns
-- **Output contract**: Commands with `--format` emit success via `agent_success_formatted(result, markdown_renderer, output_format)` — JSON or a markdown renderer per the explicit `output_format` argument. `agent_error(message, error_type, cause)` dispatches JSON or markdown by reading the `_agent_cli_error_format` ContextVar, which each `--format`-aware command sets via `set_agent_cli_error_format(error_format or output_format)` at function entry. Exceptions that print directly to stdout: `fmt`/`lint` (plxt passthrough), `concept`/`pipe` (raw TOML). Markdown renderers: `format_run_markdown` (`run/_output_helpers.py`), `format_validate_markdown` (`validate/_output_helpers.py`), `_format_init_markdown` (`init_cmd.py`).
+- **Output contract**: Commands with `--format` emit success via `agent_success_formatted(result, markdown_renderer, output_format)` — JSON or a markdown renderer per the explicit `output_format` argument. `agent_error(message, error_type, cause)` dispatches JSON or markdown by reading the `_agent_cli_error_format` ContextVar, which each `--format`-aware command sets via `set_agent_cli_error_format(error_format or output_format)` at function entry. Exceptions that print directly to stdout: `fmt`/`lint` (plxt passthrough), `concept`/`pipe` (raw TOML). Markdown renderers: `format_run_markdown` (`run/_output_helpers.py`), `format_validate_markdown` (lifted to the non-CLI `pipelex/pipeline/validation_render.py` so `pipelex-api` can import it without CLI/Typer deps), `_format_init_markdown` (`init_cmd.py`).
- **Error classification**: Each error type maps to a domain (`input`, `config`, `runtime`), a hint string, and a `retryable` flag. See `AGENT_ERROR_HINTS` dict in `agent_output.py`. The `error_domain` also drives the HTTP-status mapping for downstream APIs — see `error_domain_to_http_status()` in `pipelex/base_exceptions.py`.
- **Init**: All commands that need Pipelex use `make_pipelex_for_agent_cli(library_dirs)`. It catches init errors and routes them through `agent_error()`.
- **Async core**: Run and validate are async — commands use `asyncio.run()`.
diff --git a/pipelex/cli/agent_cli/commands/agent_output.py b/pipelex/cli/agent_cli/commands/agent_output.py
index b0a14d1cd..7f5554126 100644
--- a/pipelex/cli/agent_cli/commands/agent_output.py
+++ b/pipelex/cli/agent_cli/commands/agent_output.py
@@ -303,15 +303,15 @@ def _render_error_markdown(payload: dict[str, Any]) -> str:
return "\n".join(lines)
-def _agent_error_json(message: str, *, error_type: str, cause: BaseException | None, extra: dict[str, Any]) -> NoReturn:
- """Print a structured JSON error to stderr and exit with code 1."""
+def _agent_error_json(message: str, *, error_type: str, cause: BaseException | None, extra: dict[str, Any], exit_code: int = 1) -> NoReturn:
+ """Print a structured JSON error to stderr and exit with ``exit_code`` (default 1)."""
payload = _assemble_error_payload(message, error_type=error_type, cause=cause, extra=extra)
print(clean_json_dumps(payload, indent=2), file=sys.stderr)
- raise typer.Exit(1) from cause
+ raise typer.Exit(exit_code) from cause
-def agent_error_markdown(message: str, *, error_type: str, cause: BaseException | None = None, **extra: Any) -> NoReturn:
- """Print a markdown-rendered error to stderr and exit with code 1.
+def agent_error_markdown(message: str, *, error_type: str, cause: BaseException | None = None, exit_code: int = 1, **extra: Any) -> NoReturn:
+ """Print a markdown-rendered error to stderr and exit with ``exit_code`` (default 1).
The markdown sibling of :func:`agent_error`'s JSON path: an error-type
heading, the message body, the hint as a tip callout, and structured fields
@@ -323,15 +323,18 @@ def agent_error_markdown(message: str, *, error_type: str, cause: BaseException
message: Human-readable error message.
error_type: Error class name for programmatic matching.
cause: Optional exception to chain with ``raise ... from``.
+ exit_code: Process exit code. 1 (default) marks a produced negative
+ verdict; 2 marks a no-verdict condition (bad args, unresolvable
+ target, setup error) — the validate surface's 0/1/2 policy.
**extra: Additional fields merged into the payload.
"""
payload = _assemble_error_payload(message, error_type=error_type, cause=cause, extra=extra)
print(_render_error_markdown(payload), file=sys.stderr)
- raise typer.Exit(1) from cause
+ raise typer.Exit(exit_code) from cause
-def agent_error(message: str, *, error_type: str, cause: BaseException | None = None, **extra: Any) -> NoReturn:
- """Emit a structured error to stderr and exit with code 1.
+def agent_error(message: str, *, error_type: str, cause: BaseException | None = None, exit_code: int = 1, **extra: Any) -> NoReturn:
+ """Emit a structured error to stderr and exit with ``exit_code`` (default 1).
Dispatches on the active error format (see :func:`set_agent_cli_error_format`):
JSON by default, markdown when a command has opted in via ``--format`` or
@@ -343,14 +346,17 @@ def agent_error(message: str, *, error_type: str, cause: BaseException | None =
message: Human-readable error message.
error_type: Error class name for programmatic matching.
cause: Optional exception to chain with ``raise ... from``.
+ exit_code: Process exit code. 1 (default) marks a produced negative
+ verdict; the validate commands pass 2 for no-verdict conditions
+ (bad args, unresolvable target, setup error) per the 0/1/2 policy.
**extra: Additional fields merged into the payload.
Can override the auto-added ``hint`` field.
"""
match get_agent_cli_error_format():
case CliOutputFormat.JSON:
- _agent_error_json(message, error_type=error_type, cause=cause, extra=extra)
+ _agent_error_json(message, error_type=error_type, cause=cause, extra=extra, exit_code=exit_code)
case CliOutputFormat.MARKDOWN:
- agent_error_markdown(message, error_type=error_type, cause=cause, **extra)
+ agent_error_markdown(message, error_type=error_type, cause=cause, exit_code=exit_code, **extra)
def agent_success(result: dict[str, Any]) -> None:
diff --git a/pipelex/cli/agent_cli/commands/validate/_output_helpers.py b/pipelex/cli/agent_cli/commands/validate/_output_helpers.py
deleted file mode 100644
index e032a59c8..000000000
--- a/pipelex/cli/agent_cli/commands/validate/_output_helpers.py
+++ /dev/null
@@ -1,75 +0,0 @@
-"""Shared output formatting helpers for agent CLI validate commands."""
-
-from __future__ import annotations
-
-from typing import Any, cast
-
-
-def format_validate_markdown(result: dict[str, Any]) -> str:
- """Render a validation result dict as agent-readable markdown.
-
- Covers the success-path shape produced by the ``validate`` core helpers:
- ``validated_pipes`` (a list of ``{pipe_ref, status}``), ``total_pipes``,
- and an optional ``bundle_path``. ``validate bundle --graph`` / ``--view``
- also merge ``graph_files`` / ``graphspec`` into the result — those are
- surfaced too so markdown mode does not silently drop them.
-
- Args:
- result: The JSON-mode result dict for the validation.
-
- Returns:
- A markdown string for stdout.
- """
- lines: list[str] = ["# Validation passed", ""]
-
- bundle_path = result.get("bundle_path")
- if bundle_path:
- lines += [f"**Bundle:** `{bundle_path}`", ""]
-
- validated_pipes: list[dict[str, Any]] = result.get("validated_pipes") or []
- total_pipes = result.get("total_pipes", len(validated_pipes))
- lines.append(f"Validated {total_pipes} pipe(s):")
- lines.append("")
- for entry in validated_pipes:
- lines.append(f"- `{entry.get('pipe_ref')}` — {entry.get('status')}")
-
- # Runnability verdict — gated on key *presence*, not truthiness. Only the bundle-validate
- # surfaces put `pending_signatures` in the result; `validate all` / `validate pipe` omit the key,
- # so they get no verdict (claiming runnability there would be misleading). Present-and-empty is a
- # complete bundle (runnable); present-and-non-empty still has forward declarations to implement.
- if "pending_signatures" in result:
- pending_signatures: list[str] = result["pending_signatures"] or []
- if pending_signatures:
- # Verdict line is emitted immediately above the verbatim "Pending signatures" heading —
- # downstream consumers rely on that exact ordering (see agent_cli/CLAUDE.md).
- lines += [
- "",
- (
- f"⚠️ This method is NOT yet runnable — {len(pending_signatures)} pipe(s) are still "
- "`PipeSignature` placeholders and must be implemented before running:"
- ),
- "",
- f"## Pending signatures ({len(pending_signatures)})",
- "",
- ]
- for pending_ref in pending_signatures:
- lines.append(f"- `{pending_ref}`")
- else:
- lines += [
- "",
- (
- "✅ All pipes are concretely implemented — no `PipeSignature` placeholders remain. "
- "Strict validation will pass; this method is runnable."
- ),
- ]
-
- graph_files = result.get("graph_files")
- if isinstance(graph_files, dict):
- lines += ["", "## Graph files", ""]
- for graph_label, graph_path in cast("dict[str, Any]", graph_files).items():
- lines.append(f"- **{graph_label}:** `{graph_path}`")
-
- if "graphspec" in result:
- lines += ["", "_A GraphSpec (structured graph JSON) is included — use `--format json` to retrieve it._"]
-
- return "\n".join(lines)
diff --git a/pipelex/cli/agent_cli/commands/validate/bundle_cmd.py b/pipelex/cli/agent_cli/commands/validate/bundle_cmd.py
index 33bf861b6..7c3f04cfc 100644
--- a/pipelex/cli/agent_cli/commands/validate/bundle_cmd.py
+++ b/pipelex/cli/agent_cli/commands/validate/bundle_cmd.py
@@ -15,7 +15,6 @@ from pipelex.cli.agent_cli.commands.agent_output import (
extract_validation_errors,
set_agent_cli_error_format,
)
-from pipelex.cli.agent_cli.commands.validate._output_helpers import format_validate_markdown
from pipelex.cli.agent_cli.commands.validate._validate_core import (
validate_bundle_core,
validate_pipe_in_bundle_core,
@@ -28,6 +27,7 @@ from pipelex.libraries.pipe.exceptions import PipeNotFoundError
from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityError
from pipelex.pipelex import Pipelex
from pipelex.pipeline.exceptions import PipelineExecutionError, ValidateBundleError
+from pipelex.pipeline.validation_render import format_validate_markdown
from pipelex.tools.misc.chart_utils import FlowchartDirection
@@ -102,13 +102,14 @@ def validate_bundle_cmd(
else:
mthds_files = list(target_path.glob(f"*{MTHDS_EXTENSION}"))
if len(mthds_files) == 0:
- agent_error(f"No .mthds bundle file found in directory '{path}'", error_type="FileNotFoundError")
+ agent_error(f"No .mthds bundle file found in directory '{path}'", error_type="FileNotFoundError", exit_code=2)
if len(mthds_files) > 1:
mthds_names = ", ".join(mthds_file.name for mthds_file in mthds_files)
agent_error(
f"Multiple .mthds files found in '{path}' ({mthds_names}) and no '{DEFAULT_BUNDLE_FILE_NAME}'. "
f"Pass the .mthds file directly instead.",
error_type="ArgumentError",
+ exit_code=2,
)
bundle_path = str(mthds_files[0])
@@ -126,6 +127,7 @@ def validate_bundle_cmd(
f"'{path}' is not a .mthds file or directory. "
f"Use 'validate pipe <code>' for pipe codes, or 'validate bundle <path>' for .mthds files/directories.",
error_type="ArgumentError",
+ exit_code=2,
)
library_dirs = [Path(lib_dir) for lib_dir in library_dir] if library_dir else None
@@ -169,14 +171,14 @@ def validate_bundle_cmd(
if exc.__cause__:
graph_extra["cause_type"] = type(exc.__cause__).__name__
graph_extra["cause_message"] = str(exc.__cause__)
- agent_error(f"Graph generation failed: {exc.message}", error_type="PipelineExecutionError", cause=exc, **graph_extra)
+ agent_error(f"Graph generation failed: {exc.message}", error_type="PipelineExecutionError", cause=exc, exit_code=2, **graph_extra)
except PipelexInterpreterError as exc:
- agent_error(f"Graph generation failed: {exc}", error_type=type(exc).__name__, cause=exc)
+ agent_error(f"Graph generation failed: {exc}", error_type=type(exc).__name__, cause=exc, exit_code=2)
except typer.Exit:
raise
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(f"Graph generation failed: {exc}", error_type=type(exc).__name__, cause=exc)
+ agent_error(f"Graph generation failed: {exc}", error_type=type(exc).__name__, cause=exc, exit_code=2)
# Generate view (GraphSpec JSON) if requested and validation succeeded
if view:
@@ -197,14 +199,14 @@ def validate_bundle_cmd(
if exc.__cause__:
view_extra["cause_type"] = type(exc.__cause__).__name__
view_extra["cause_message"] = str(exc.__cause__)
- agent_error(f"View generation failed: {exc.message}", error_type="PipelineExecutionError", cause=exc, **view_extra)
+ agent_error(f"View generation failed: {exc.message}", error_type="PipelineExecutionError", cause=exc, exit_code=2, **view_extra)
except PipelexInterpreterError as exc:
- agent_error(f"View generation failed: {exc}", error_type=type(exc).__name__, cause=exc)
+ agent_error(f"View generation failed: {exc}", error_type=type(exc).__name__, cause=exc, exit_code=2)
except typer.Exit:
raise
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(f"View generation failed: {exc}", error_type=type(exc).__name__, cause=exc)
+ agent_error(f"View generation failed: {exc}", error_type=type(exc).__name__, cause=exc, exit_code=2)
agent_success_formatted(result, markdown_renderer=format_validate_markdown, output_format=output_format)
@@ -222,10 +224,10 @@ def validate_bundle_cmd(
raise typer.Exit(1)
except PipeNotFoundError as exc:
- agent_error(str(exc), error_type="PipeNotFoundError", cause=exc)
+ agent_error(str(exc), error_type="PipeNotFoundError", cause=exc, exit_code=2)
except FileNotFoundError as exc:
- agent_error(f"Bundle file not found: {bundle_path}", error_type="FileNotFoundError", cause=exc)
+ agent_error(f"Bundle file not found: {bundle_path}", error_type="FileNotFoundError", cause=exc, exit_code=2)
except ValidateBundleError as exc:
# Invalid verdict: emit the structured failure envelope. validation_errors[] is the shared
@@ -246,6 +248,7 @@ def validate_bundle_cmd(
exc.message,
error_type="PipeOperatorModelChoiceError",
cause=exc,
+ exit_code=2,
pipe_code=exc.pipe_code,
model_type=str(exc.model_type),
model_choice=str(exc.model_choice),
@@ -260,14 +263,14 @@ def validate_bundle_cmd(
availability_extra["fallback_list"] = exc.fallback_list
if exc.pipe_stack:
availability_extra["pipe_stack"] = exc.pipe_stack
- agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, **availability_extra)
+ agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, exit_code=2, **availability_extra)
except typer.Exit:
raise
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(str(exc), error_type=type(exc).__name__, cause=exc)
+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc, exit_code=2)
finally:
Pipelex.teardown_if_needed()
diff --git a/pipelex/cli/agent_cli/commands/validate/method_cmd.py b/pipelex/cli/agent_cli/commands/validate/method_cmd.py
index e7411dbcb..b4b1eb8f2 100644
--- a/pipelex/cli/agent_cli/commands/validate/method_cmd.py
+++ b/pipelex/cli/agent_cli/commands/validate/method_cmd.py
@@ -16,7 +16,6 @@ from pipelex.cli.agent_cli.commands.agent_output import (
extract_validation_errors,
set_agent_cli_error_format,
)
-from pipelex.cli.agent_cli.commands.validate._output_helpers import format_validate_markdown
from pipelex.cli.agent_cli.commands.validate._validate_core import (
validate_bundle_core,
validate_pipe_in_bundle_core,
@@ -26,6 +25,7 @@ from pipelex.core.pipes.exceptions import PipeOperatorModelChoiceError
from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityError
from pipelex.pipelex import Pipelex
from pipelex.pipeline.exceptions import ValidateBundleError
+from pipelex.pipeline.validation_render import format_validate_markdown
def validate_method_cmd(
@@ -76,7 +76,7 @@ def validate_method_cmd(
library_dirs=library_dir,
)
if not method.mthds_files:
- agent_error(f"Method '{name}' has no .mthds bundle files.", error_type="MethodError")
+ agent_error(f"Method '{name}' has no .mthds bundle files.", error_type="MethodError", exit_code=2)
bundle_path = method.mthds_files[0]
@@ -114,7 +114,7 @@ def validate_method_cmd(
raise typer.Exit(1)
except FileNotFoundError as exc:
- agent_error(f"Bundle file not found: {bundle_path}", error_type="FileNotFoundError", cause=exc)
+ agent_error(f"Bundle file not found: {bundle_path}", error_type="FileNotFoundError", cause=exc, exit_code=2)
except ValidateBundleError as exc:
# Invalid verdict (see bundle_cmd): structured failure envelope; validation_errors[] is the
@@ -133,6 +133,7 @@ def validate_method_cmd(
exc.message,
error_type="PipeOperatorModelChoiceError",
cause=exc,
+ exit_code=2,
pipe_code=exc.pipe_code,
model_type=str(exc.model_type),
model_choice=str(exc.model_choice),
@@ -147,7 +148,7 @@ def validate_method_cmd(
availability_extra["fallback_list"] = exc.fallback_list
if exc.pipe_stack:
availability_extra["pipe_stack"] = exc.pipe_stack
- agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, **availability_extra)
+ agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, exit_code=2, **availability_extra)
except typer.Exit:
# The runnability gate raises typer.Exit(1) after emitting the success envelope; let it
@@ -156,7 +157,7 @@ def validate_method_cmd(
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(str(exc), error_type=type(exc).__name__, cause=exc)
+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc, exit_code=2)
finally:
Pipelex.teardown_if_needed()
diff --git a/pipelex/cli/agent_cli/commands/validate/pipe_cmd.py b/pipelex/cli/agent_cli/commands/validate/pipe_cmd.py
index 0385bdfb6..ab9f230a5 100644
--- a/pipelex/cli/agent_cli/commands/validate/pipe_cmd.py
+++ b/pipelex/cli/agent_cli/commands/validate/pipe_cmd.py
@@ -14,7 +14,6 @@ from pipelex.cli.agent_cli.commands.agent_output import (
extract_validation_errors,
set_agent_cli_error_format,
)
-from pipelex.cli.agent_cli.commands.validate._output_helpers import format_validate_markdown
from pipelex.cli.agent_cli.commands.validate._validate_core import (
validate_all_core,
validate_pipe_core,
@@ -24,8 +23,10 @@ from pipelex.core.interpreter.helpers import MTHDS_EXTENSION, is_pipelex_file
from pipelex.core.pipes.exceptions import PipeOperatorModelChoiceError
from pipelex.libraries.pipe.exceptions import PipeNotFoundError
from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityError
+from pipelex.pipe_run.exceptions import DryRunError
from pipelex.pipelex import Pipelex
from pipelex.pipeline.exceptions import ValidateBundleError
+from pipelex.pipeline.validation_render import format_validate_markdown
def validate_pipe_cmd(
@@ -75,7 +76,7 @@ def validate_pipe_cmd(
# Handle --all flag
if validate_all:
if pipe_code:
- agent_error("--all cannot be used with a pipe code", error_type="ArgumentError")
+ agent_error("--all cannot be used with a pipe code", error_type="ArgumentError", exit_code=2)
make_pipelex_for_agent_cli(library_dirs=library_dirs, needs_inference=False, needs_model_specs=True)
@@ -111,6 +112,7 @@ def validate_pipe_cmd(
pipe_code=exc.pipe_code,
model_type=str(exc.model_type),
model_choice=str(exc.model_choice),
+ exit_code=2,
)
except PipeOperatorModelAvailabilityError as exc:
@@ -120,8 +122,15 @@ def validate_pipe_cmd(
cause=exc,
pipe_code=exc.pipe_code,
model_handle=exc.model_handle,
+ exit_code=2,
)
+ except DryRunError as exc:
+ # A dry-run failure is a produced NEGATIVE VERDICT (a pipe is invalid) — exit 1, NOT the
+ # catch-all's no-verdict 2. `validate --all` sweeps via validate_current_library, which
+ # raises DryRunError directly (not wrapped in ValidateBundleError as the bundle path is).
+ agent_error(str(exc), error_type="DryRunError", cause=exc, is_valid=False, exit_code=1)
+
except typer.Exit:
# The runnability gate raises typer.Exit(1) after emitting the success envelope; let it
# propagate (exit code) rather than be reshaped into an agent_error by the broad handler below.
@@ -129,7 +138,7 @@ def validate_pipe_cmd(
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(str(exc), error_type=type(exc).__name__, cause=exc)
+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc, exit_code=2)
finally:
Pipelex.teardown_if_needed()
@@ -139,6 +148,7 @@ def validate_pipe_cmd(
agent_error(
"No pipe code specified. Use --all to validate all pipes, or use 'validate bundle <path>' for bundle files.",
error_type="ArgumentError",
+ exit_code=2,
)
# Helpful error if the user passes a path instead of a pipe code
@@ -148,6 +158,7 @@ def validate_pipe_cmd(
f"'{pipe_code}' looks like a file path or directory. "
f"Use 'validate bundle {pipe_code}' for bundles/directories, or 'validate pipe <code>' for pipe codes.",
error_type="ArgumentError",
+ exit_code=2,
)
# Check installed methods' exports for additional library dirs
@@ -158,6 +169,7 @@ def validate_pipe_cmd(
f"Ambiguous pipe code '{pipe_code}': {exc}",
error_type="ArgumentError",
cause=exc,
+ exit_code=2,
)
if export_dirs:
export_paths = [Path(export_dir) for export_dir in export_dirs]
@@ -176,7 +188,7 @@ def validate_pipe_cmd(
error_message = str(exc)
if pipe_code == "all":
error_message += " Did you mean '--all'?"
- agent_error(error_message, error_type="PipeNotFoundError", cause=exc)
+ agent_error(error_message, error_type="PipeNotFoundError", cause=exc, exit_code=2)
except ValidateBundleError as exc:
# Invalid verdict (see the --all arm): structured failure envelope; validation_errors[] is the
@@ -197,6 +209,7 @@ def validate_pipe_cmd(
pipe_code=exc.pipe_code,
model_type=str(exc.model_type),
model_choice=str(exc.model_choice),
+ exit_code=2,
)
except PipeOperatorModelAvailabilityError as exc:
@@ -208,14 +221,20 @@ def validate_pipe_cmd(
availability_extra["fallback_list"] = exc.fallback_list
if exc.pipe_stack:
availability_extra["pipe_stack"] = exc.pipe_stack
- agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, **availability_extra)
+ agent_error(exc.message, error_type="PipeOperatorModelAvailabilityError", cause=exc, **availability_extra, exit_code=2)
+
+ except DryRunError as exc:
+ # A dry-run failure is a produced NEGATIVE VERDICT (the pipe is invalid) — exit 1, NOT the
+ # catch-all's no-verdict 2. validate_pipe_core sweeps via validate_pipes, which raises
+ # DryRunError directly (not wrapped in ValidateBundleError as the bundle path is).
+ agent_error(str(exc), error_type="DryRunError", cause=exc, is_valid=False, exit_code=1)
except typer.Exit:
raise
except Exception as exc: # noqa: BLE001
# Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.
- agent_error(str(exc), error_type=type(exc).__name__, cause=exc)
+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc, exit_code=2)
finally:
Pipelex.teardown_if_needed()
diff --git a/pipelex/cli/commands/validate/_validate_core.py b/pipelex/cli/commands/validate/_validate_core.py
index d025c77f2..bd28f812f 100644
--- a/pipelex/cli/commands/validate/_validate_core.py
+++ b/pipelex/cli/commands/validate/_validate_core.py
@@ -36,6 +36,7 @@ from pipelex.pipeline.validate_bundle import build_pending_signatures, validate_
from pipelex.system.runtime import IntegrationMode
from pipelex.system.telemetry.events import EventProperty
from pipelex.tools.misc.package_utils import get_package_version
+from pipelex.tools.misc.string_utils import count_with_noun
if TYPE_CHECKING:
from pathlib import Path
@@ -78,7 +79,7 @@ def do_validate_all_libraries_and_dry_run(
pipes = all_pipes if allow_signatures else [pipe for pipe in all_pipes if not pipe.is_signature]
if library_dirs:
dirs_str = ", ".join(f'"{lib_dir}"' for lib_dir in library_dirs)
- typer.echo(f"Validating {len(pipes)} pipe(s) from: {dirs_str}")
+ typer.echo(f"Validating {count_with_noun(len(pipes), singular='pipe')} from: {dirs_str}")
# validate_current_library owns the static wiring pass and the single PIPE_DRY_RUN telemetry
# event — sweeping the library we just loaded, without teardown.
@@ -103,9 +104,9 @@ def do_validate_all_libraries_and_dry_run(
signature_count = sum(1 for pipe in all_pipes if pipe.is_signature)
typer.echo(f"Setup sequence passed OK, config and pipelines are validated.{_format_signatures_summary_suffix(signature_count)}")
except PipeOperatorModelAvailabilityError as exc:
- handle_model_availability_error(exc, context=ErrorContext.VALIDATION)
+ handle_model_availability_error(exc, context=ErrorContext.VALIDATION, exit_code=2)
except PipeOperatorModelChoiceError as exc:
- handle_model_choice_error(exc, context=ErrorContext.VALIDATION)
+ handle_model_choice_error(exc, context=ErrorContext.VALIDATION, exit_code=2)
async def _validate_pipe_or_bundle(
@@ -147,7 +148,7 @@ async def _validate_pipe_or_bundle(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1) from exc
+ raise typer.Exit(2) from exc
except ValidateBundleError as bundle_error:
handle_validate_bundle_error(bundle_error, bundle_path=bundle_path)
elif pipe_code:
@@ -180,7 +181,7 @@ async def _validate_pipe_or_bundle(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
def execute_validate(
@@ -228,10 +229,10 @@ def execute_validate(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1) from exc
+ raise typer.Exit(2) from exc
except PipeOperatorModelChoiceError as exc:
- handle_model_choice_error(exc, context=ErrorContext.VALIDATION)
+ handle_model_choice_error(exc, context=ErrorContext.VALIDATION, exit_code=2)
except PipeOperatorModelAvailabilityError as exc:
- handle_model_availability_error(exc, context=ErrorContext.VALIDATION)
+ handle_model_availability_error(exc, context=ErrorContext.VALIDATION, exit_code=2)
finally:
Pipelex.teardown_if_needed()
diff --git a/pipelex/cli/commands/validate/bundle_cmd.py b/pipelex/cli/commands/validate/bundle_cmd.py
index 4e125f579..6b780608e 100644
--- a/pipelex/cli/commands/validate/bundle_cmd.py
+++ b/pipelex/cli/commands/validate/bundle_cmd.py
@@ -62,7 +62,7 @@ def validate_bundle_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
if len(mthds_files) > 1:
mthds_names = ", ".join(mthds_file.name for mthds_file in mthds_files)
typer.secho(
@@ -72,7 +72,7 @@ def validate_bundle_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
bundle_path = str(mthds_files[0])
# Add directory as library dir
@@ -94,7 +94,7 @@ def validate_bundle_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
library_dirs_paths = [Path(lib_dir) for lib_dir in library_dir] if library_dir else None
diff --git a/pipelex/cli/commands/validate/pipe_cmd.py b/pipelex/cli/commands/validate/pipe_cmd.py
index dade273ec..f11dd8926 100644
--- a/pipelex/cli/commands/validate/pipe_cmd.py
+++ b/pipelex/cli/commands/validate/pipe_cmd.py
@@ -65,7 +65,7 @@ def validate_pipe_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
library_dirs_paths = [Path(lib_dir) for lib_dir in library_dir] if library_dir else None
try:
make_pipelex_for_cli(
@@ -90,7 +90,7 @@ def validate_pipe_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
# Helpful error if the user passes a path instead of a pipe code
target_path = Path(pipe_code)
@@ -101,7 +101,7 @@ def validate_pipe_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1)
+ raise typer.Exit(2)
# Check installed methods' exports for additional library dirs
try:
@@ -112,7 +112,7 @@ def validate_pipe_cmd(
fg=typer.colors.RED,
err=True,
)
- raise typer.Exit(1) from exc
+ raise typer.Exit(2) from exc
if extra_dirs:
if library_dir is None:
library_dir = extra_dirs
diff --git a/pipelex/cli/error_handlers.py b/pipelex/cli/error_handlers.py
index 5d8d7645d..e37a1ad06 100644
--- a/pipelex/cli/error_handlers.py
+++ b/pipelex/cli/error_handlers.py
@@ -121,12 +121,14 @@ def display_error_panel(
console.print()
-def handle_model_choice_error(exc: PipeOperatorModelChoiceError, *, context: ErrorContext) -> NoReturn:
+def handle_model_choice_error(exc: PipeOperatorModelChoiceError, *, context: ErrorContext, exit_code: int = 1) -> NoReturn:
"""Handle and display PipeOperatorModelChoiceError with formatted output.
Args:
exc: The model choice error exception
context: Context for the error message
+ exit_code: Process exit code. The validate surface passes 2 (a no-verdict
+ setup/config error per its 0/1/2 policy); other contexts keep the default 1.
"""
console = get_console()
print_traceback_if_requested(console)
@@ -149,15 +151,17 @@ def handle_model_choice_error(exc: PipeOperatorModelChoiceError, *, context: Err
("Join our Discord for help", URLs.discord),
],
)
- raise typer.Exit(1) from exc
+ raise typer.Exit(exit_code) from exc
-def handle_model_availability_error(exc: PipeOperatorModelAvailabilityError, *, context: ErrorContext) -> NoReturn:
+def handle_model_availability_error(exc: PipeOperatorModelAvailabilityError, *, context: ErrorContext, exit_code: int = 1) -> NoReturn:
"""Handle and display PipeOperatorModelAvailabilityError with formatted output.
Args:
exc: The model availability error exception
context: Context for the error message
+ exit_code: Process exit code. The validate surface passes 2 (a no-verdict
+ setup/config error per its 0/1/2 policy); other contexts keep the default 1.
"""
console = get_console()
print_traceback_if_requested(console)
@@ -186,7 +190,7 @@ def handle_model_availability_error(exc: PipeOperatorModelAvailabilityError, *,
("Join our Discord for help", URLs.discord),
],
)
- raise typer.Exit(1) from exc
+ raise typer.Exit(exit_code) from exc
def handle_model_deck_preset_error(exc: ModelDeckPresetValidatonError, *, context: ErrorContext) -> NoReturn:
diff --git a/pipelex/core/exceptions.py b/pipelex/core/exceptions.py
index 115bb28c4..af5e2d601 100644
--- a/pipelex/core/exceptions.py
+++ b/pipelex/core/exceptions.py
@@ -44,6 +44,9 @@ class PipesAndConceptValidationErrorData(BaseModel):
# === Entity Context (what failed) ===
pipe_code: str | None = Field(default=None, description="Pipe code if error is in a pipe")
concept_code: str | None = Field(default=None, description="Concept code if error is in a concept")
+ missing_pipe_code: str | None = Field(
+ default=None, description="Referenced pipe code that does not resolve (for unresolved pipe-dependency errors)"
+ )
field_name: str | None = Field(default=None, description="Specific field that failed")
# === Error Classification ===
diff --git a/pipelex/core/interpreter/validation_error_categorizer.py b/pipelex/core/interpreter/validation_error_categorizer.py
index 0802937a7..01caf68ce 100644
--- a/pipelex/core/interpreter/validation_error_categorizer.py
+++ b/pipelex/core/interpreter/validation_error_categorizer.py
@@ -189,9 +189,11 @@ def categorize_blueprint_validation_error(
loc = error["loc"]
message = error["msg"]
- # Extract pipe code from location if available (e.g., ('pipes', 'extract_details_of_task', ...))
+ # Extract pipe code from location if available (e.g., ('pipe', 'extract_details_of_task', ...)).
+ # The blueprint field is `pipe` (singular), so the loc key is "pipe" — the previous "pipes"
+ # never matched, silently dropping the pipe_code locator from every categorized pipe item.
pipe_code: str | None = None
- if len(loc) >= 2 and loc[0] == "pipes":
+ if len(loc) >= 2 and loc[0] == "pipe":
pipe_code = str(loc[1])
error_scope = get_error_scope(loc)
@@ -210,6 +212,18 @@ def categorize_blueprint_validation_error(
case ValidationErrorScope.PIPE | ValidationErrorScope.DOMAIN | ValidationErrorScope.MAIN_PIPE | ValidationErrorScope.BUNDLE:
pass
+ # Unknown pipe `type`: a pydantic discriminated-union tag failure (the pipe declared a `type`
+ # matching no known pipe operator/controller). Carry it as a categorized blueprint item with the
+ # pipe_code locator instead of letting it fall through to a bare residual.
+ if error["type"] == "union_tag_invalid" and pipe_code is not None:
+ return PipelexBundleBlueprintValidationErrorData(
+ error_type=PipeValidationErrorType.UNKNOWN_PIPE_TYPE,
+ domain_code=domain,
+ source=source,
+ pipe_code=pipe_code,
+ message=message,
+ )
+
# Try to categorize input validation errors (missing/unused inputs)
input_error = _categorize_input_validation_error(
message=message,
diff --git a/pipelex/core/pipes/exceptions.py b/pipelex/core/pipes/exceptions.py
index a5f4ad3f9..ba03e0faa 100644
--- a/pipelex/core/pipes/exceptions.py
+++ b/pipelex/core/pipes/exceptions.py
@@ -111,8 +111,14 @@ class PipeValidationErrorType(StrEnum):
LLM_OUTPUT_CANNOT_BE_IMAGE = "llm_output_cannot_be_image"
IMG_GEN_INPUT_NOT_TEXT_COMPATIBLE = "img_gen_input_not_text_compatible"
INVALID_PIPE_CODE_SYNTAX = "invalid_pipe_code_syntax"
+ UNKNOWN_PIPE_TYPE = "unknown_pipe_type"
BATCH_ITEM_NAME_COLLISION = "batch_item_name_collision"
+ # Wiring / reference-resolution failures, detected when validating a pipe's contract against the
+ # merged library (a referenced concept or dependency pipe does not resolve).
+ UNRESOLVED_CONCEPT = "unresolved_concept"
+ UNRESOLVED_PIPE_DEPENDENCY = "unresolved_pipe_dependency"
+
# Generic fallback for unexpected validation errors
UNKNOWN_VALIDATION_ERROR = "unknown_validation_error"
diff --git a/pipelex/libraries/concept/exceptions.py b/pipelex/libraries/concept/exceptions.py
index f0d412c31..f779a0b7c 100644
--- a/pipelex/libraries/concept/exceptions.py
+++ b/pipelex/libraries/concept/exceptions.py
@@ -1,5 +1,14 @@
-from pipelex.libraries.exceptions import LibraryError
+from pipelex.libraries.exceptions import LibraryLoadingError
-class ConceptLibraryError(LibraryError):
- pass
+class ConceptLibraryError(LibraryLoadingError):
+ """A concept-library failure (unresolved / duplicate / invalid concept reference).
+
+ Extends ``LibraryLoadingError`` so it can carry the structured
+ ``pipe_concept_validation_errors`` that the bundle-validation error cascade
+ forwards onto ``validation_errors[]`` — an unresolved concept reference then
+ surfaces as a categorized ``pipe_validation`` item (with the referencing
+ pipe/field and the missing concept) rather than a bare message-only residual.
+ Call sites that only pass a message keep working: the structured lists default
+ to empty, so those raises behave exactly as before (one fallback residual).
+ """
diff --git a/pipelex/libraries/concept_reference_validation.py b/pipelex/libraries/concept_reference_validation.py
index 85ed93513..30fe7d06c 100644
--- a/pipelex/libraries/concept_reference_validation.py
+++ b/pipelex/libraries/concept_reference_validation.py
@@ -1,10 +1,32 @@
+from pipelex.core.bundles.exceptions import PipelexBundleBlueprintValidationErrorData
from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint
from pipelex.core.concepts.concept_factory import ConceptFactory
from pipelex.core.concepts.native.concept_native import NativeConceptCode
+from pipelex.core.exceptions import PipesAndConceptValidationErrorData
+from pipelex.core.pipes.exceptions import PipeValidationErrorType
from pipelex.core.qualified_ref import QualifiedRef
from pipelex.libraries.concept.exceptions import ConceptLibraryError
+def _split_concept_ref_context(context: str) -> tuple[str | None, str | None, str]:
+ """Split a concept-reference context path into ``(owning_pipe_code, owning_concept_code, field_name)``.
+
+ The context paths produced by ``PipelexBundleBlueprint.collect_concept_references``
+ look like ``pipe.<code>.output``, ``pipe.<code>.inputs.<name>``, ``concept.<code>.refines``,
+ or ``concept.<code>.structure.<field>.concept_ref``. Returns the owning pipe code (set when a
+ pipe owns the reference), the owning concept code (set when a concept owns it), and the field
+ path relative to the owner (everything after ``pipe.<code>.`` / ``concept.<code>.``). At most
+ one of the two owner codes is non-``None``. The full path is kept separately as ``field_path``.
+ """
+ parts = context.split(".")
+ if len(parts) >= 3 and parts[0] in {"pipe", "concept"}:
+ field_name = ".".join(parts[2:])
+ if parts[0] == "pipe":
+ return parts[1], None, field_name
+ return None, parts[1], field_name
+ return None, None, context
+
+
def validate_concept_references_in_blueprints(
blueprints: list[PipelexBundleBlueprint],
*,
@@ -45,6 +67,13 @@ def validate_concept_references_in_blueprints(
native_codes = {native.value for native in NativeConceptCode.values_list()}
undeclared: list[str] = []
+ # Pipe-owned references (a pipe input/output) and concept-owned references (a concept's
+ # `refines` or a structure field's `concept_ref`) are categorized differently: a pipe-owned
+ # miss is a `pipe_validation` item, a concept-owned miss is a `blueprint_validation` item
+ # (consistent with how every other concept error is categorized), so we never emit a phantom
+ # `pipe_validation` item with a null pipe_code for a concept-level failure.
+ unresolved_pipe_items: list[PipesAndConceptValidationErrorData] = []
+ unresolved_concept_items: list[PipelexBundleBlueprintValidationErrorData] = []
for blueprint in blueprints:
domain_code = blueprint.domain
source = blueprint.source
@@ -66,6 +95,38 @@ def validate_concept_references_in_blueprints(
concept_ref = ConceptFactory.make_concept_ref_with_domain(domain_code=domain_code, concept_code=ref.local_code)
if concept_ref not in resolvable_concept_refs:
undeclared.append(f"'{concept_ref_or_code}' in {context} is not declared in domain '{domain_code}' (source: '{source or 'unknown'}')")
+ owning_pipe_code, owning_concept_code, field_name = _split_concept_ref_context(context)
+ item_message = f"Concept '{concept_ref_or_code}' in {context} is not declared in domain '{domain_code}' and is not native."
+ if owning_pipe_code is not None:
+ # Pipe-owned reference → pipe_validation, locating the referencing pipe, the
+ # missing concept (concept_code), and the field that holds the reference.
+ unresolved_pipe_items.append(
+ PipesAndConceptValidationErrorData(
+ error_type=PipeValidationErrorType.UNRESOLVED_CONCEPT,
+ domain_code=domain_code,
+ source=source,
+ pipe_code=owning_pipe_code,
+ concept_code=concept_ref_or_code,
+ field_name=field_name,
+ message=item_message,
+ field_path=context,
+ )
+ )
+ else:
+ # Concept-owned reference → blueprint_validation, consistent with how every other
+ # concept error is categorized. concept_code locates the OWNING concept (the entity
+ # in the bundle that is broken); the missing reference is named in the message.
+ # owning_concept_code is None only for a malformed context shape, in which case we
+ # fall back to the unresolved ref so no locator is lost.
+ unresolved_concept_items.append(
+ PipelexBundleBlueprintValidationErrorData(
+ error_type=PipeValidationErrorType.UNRESOLVED_CONCEPT,
+ domain_code=domain_code,
+ source=source,
+ concept_code=owning_concept_code or concept_ref_or_code,
+ message=item_message,
+ )
+ )
if undeclared:
declared_list = sorted(batch_declared_concept_refs) if batch_declared_concept_refs else "(none)"
@@ -76,4 +137,12 @@ def validate_concept_references_in_blueprints(
+ f"\nDeclared concepts: {declared_list}."
+ f"\nNative concepts: {sorted(native_codes)}."
)
- raise ConceptLibraryError(msg)
+ # ConceptLibraryError now extends LibraryLoadingError, so the structured per-reference items
+ # ride the existing `except LibraryError` forwarding in _translate_to_validate_bundle_error and
+ # surface as categorized items (not a bare residual): pipe-owned refs as `pipe_validation`,
+ # concept-owned refs as `blueprint_validation`.
+ raise ConceptLibraryError(
+ msg,
+ blueprint_validation_errors=unresolved_concept_items,
+ pipe_concept_validation_errors=unresolved_pipe_items,
+ )
diff --git a/pipelex/libraries/library.py b/pipelex/libraries/library.py
index c13dc0424..6683f6be1 100644
--- a/pipelex/libraries/library.py
+++ b/pipelex/libraries/library.py
@@ -6,11 +6,13 @@ from pydantic import BaseModel, Field, PrivateAttr
from pipelex import log
from pipelex.base_exceptions import PipelexUnexpectedError
+from pipelex.core.exceptions import PipesAndConceptValidationErrorData
+from pipelex.core.pipes.exceptions import PipeValidationErrorType
from pipelex.core.qualified_ref import QualifiedRef
from pipelex.libraries.concept.concept_library import ConceptLibrary
from pipelex.libraries.concept.exceptions import ConceptLibraryError
from pipelex.libraries.domain.domain_library import DomainLibrary
-from pipelex.libraries.exceptions import LibraryError
+from pipelex.libraries.exceptions import LibraryError, LibraryLoadingError
from pipelex.libraries.pipe.exceptions import PipeLibraryError
from pipelex.libraries.pipe.pipe_library import PipeLibrary
from pipelex.pipe_controllers.pipe_controller import PipeController
@@ -137,7 +139,20 @@ class Library(BaseModel):
self.pipe_library.get_required_pipe(pipe_code=sub_pipe_code)
except PipeLibraryError as pipe_error:
msg = f"Error validating pipe '{pipe.code}' dependency pipe '{sub_pipe_code}' because of: {pipe_error}"
- raise LibraryError(msg) from pipe_error
+ # Carry a structured item so an unresolved dependency surfaces as a categorized
+ # `pipe_validation` item: `pipe_code` is the referencing controller and
+ # `missing_pipe_code` is the dependency that does not resolve, so a machine
+ # consumer can read both without parsing the message. LibraryLoadingError rides
+ # the existing `except LibraryError` forwarding in _translate_to_validate_bundle_error.
+ dependency_error_data = PipesAndConceptValidationErrorData(
+ error_type=PipeValidationErrorType.UNRESOLVED_PIPE_DEPENDENCY,
+ domain_code=pipe.domain_code,
+ pipe_code=pipe.code,
+ missing_pipe_code=sub_pipe_code,
+ message=msg,
+ field_path=f"pipe.{pipe.code}",
+ )
+ raise LibraryLoadingError(message=msg, pipe_concept_validation_errors=[dependency_error_data]) from pipe_error
for pipe in self.pipe_library.root.values():
# Skip full validation for pipe controllers with unresolved cross-package dependencies
diff --git a/pipelex/pipeline/runner.py b/pipelex/pipeline/runner.py
index a2f7ba6aa..c95fdf036 100644
--- a/pipelex/pipeline/runner.py
+++ b/pipelex/pipeline/runner.py
@@ -335,6 +335,7 @@ class PipelexMTHDSProtocol(MTHDSProtocol["PipeOutput"]):
self,
mthds_contents: list[str],
allow_signatures: bool = False,
+ extra: dict[str, Any] | None = None,
) -> PipelexValidationReport:
"""Parse, validate, and dry-run MTHDS bundles — protocol `validate`.
@@ -366,13 +367,19 @@ class PipelexMTHDSProtocol(MTHDSProtocol["PipeOutput"]):
Args:
mthds_contents: MTHDS contents to load (always a list, even for one file).
allow_signatures: Tolerate unimplemented pipe signatures (strict by default).
+ extra: Rejected — the local runtime defines no extension args. Extension
+ args are server-specific; pass them to the server that defines them.
Returns:
PipelexValidationReport with the structural artifacts of a valid bundle.
Raises:
+ PipelineRequestError: When extension args are passed (the local runtime accepts none).
PipelexError: When the bundle is invalid (parse, static validation, or dry-run failure).
"""
+ if extra:
+ msg = f"The local runtime defines no extension args; got {sorted(extra)}."
+ raise PipelineRequestError(msg)
library_dirs = [Path(library_dir) for library_dir in self.library_dirs] if self.library_dirs else None
# Delegate to the shared in-process orchestrator (library-window management,
# graph arm, report assembly). The protocol `validate` interface carries only
diff --git a/pipelex/pipeline/validation_errors.py b/pipelex/pipeline/validation_errors.py
index b5d950b5b..533bbc321 100644
--- a/pipelex/pipeline/validation_errors.py
+++ b/pipelex/pipeline/validation_errors.py
@@ -112,6 +112,7 @@ def build_validation_error_items(
error_type=pipe_error.error_type,
pipe_code=pipe_error.pipe_code,
concept_code=pipe_error.concept_code,
+ missing_pipe_code=pipe_error.missing_pipe_code,
domain_code=pipe_error.domain_code,
source=pipe_error.source,
field_path=pipe_error.field_path or None,
diff --git a/pipelex/pipeline/validation_render.py b/pipelex/pipeline/validation_render.py
new file mode 100644
index 000000000..62d312f6c
--- /dev/null
+++ b/pipelex/pipeline/validation_render.py
@@ -0,0 +1,145 @@
+"""Public, non-CLI Markdown rendering for validation results.
+
+This is the public home of ``format_validate_markdown`` — lifted out of the
+agent CLI (``cli/agent_cli/commands/validate/_output_helpers.py``) so it can be
+imported by other surfaces (notably ``pipelex-api``'s ``/validate`` route, which
+attaches an opt-in ``rendered_markdown`` extra) WITHOUT pulling in Typer / the
+CLI internals. The agent CLI keeps using it for its ``--format markdown`` valid
+path, so the local-CLI and API-runner Markdown share one source of truth and
+cannot drift.
+
+``format_validate_markdown`` renders the **valid-arm** result;
+``render_invalid_validation_markdown`` renders the **invalid-arm** verdict (the
+structured ``validation_errors``). Both are shared pipelex code so the local CLI
+and the API-runner cannot drift in format/structure.
+"""
+
+from __future__ import annotations
+
+from typing import Any, cast
+
+from pipelex.tools.misc.string_utils import count_with_noun
+
+
+def format_validate_markdown(result: dict[str, Any]) -> str:
+ """Render a validation result dict as agent-readable markdown.
+
+ Covers the success-path shape produced by the ``validate`` core helpers:
+ ``validated_pipes`` (a list of ``{pipe_ref, status}``), ``total_pipes``,
+ and an optional ``bundle_path``. ``validate bundle --graph`` / ``--view``
+ also merge ``graph_files`` / ``graphspec`` into the result — those are
+ surfaced too so markdown mode does not silently drop them.
+
+ Args:
+ result: The JSON-mode result dict for the validation.
+
+ Returns:
+ A markdown string for stdout.
+ """
+ lines: list[str] = ["# Validation passed", ""]
+
+ bundle_path = result.get("bundle_path")
+ if bundle_path:
+ lines += [f"**Bundle:** `{bundle_path}`", ""]
+
+ validated_pipes: list[dict[str, Any]] = result.get("validated_pipes") or []
+ total_pipes = result.get("total_pipes", len(validated_pipes))
+ lines.append(f"Validated {count_with_noun(total_pipes, singular='pipe')}:")
+ lines.append("")
+ for entry in validated_pipes:
+ lines.append(f"- `{entry.get('pipe_ref')}` — {entry.get('status')}")
+
+ # Runnability verdict — gated on key *presence*, not truthiness. Only the bundle-validate
+ # surfaces put `pending_signatures` in the result; `validate all` / `validate pipe` omit the key,
+ # so they get no verdict (claiming runnability there would be misleading). Present-and-empty is a
+ # complete bundle (runnable); present-and-non-empty still has forward declarations to implement.
+ if "pending_signatures" in result:
+ pending_signatures: list[str] = result["pending_signatures"] or []
+ if pending_signatures:
+ pending_count = len(pending_signatures)
+ if pending_count == 1:
+ verdict = "⚠️ This method is NOT yet runnable — 1 pipe is still a `PipeSignature` placeholder and must be implemented before running:"
+ else:
+ verdict = (
+ f"⚠️ This method is NOT yet runnable — {pending_count} pipes are still "
+ "`PipeSignature` placeholders and must be implemented before running:"
+ )
+ # Verdict line is emitted immediately above the verbatim "Pending signatures" heading —
+ # downstream consumers rely on that exact ordering (see agent_cli/CLAUDE.md).
+ lines += [
+ "",
+ verdict,
+ "",
+ f"## Pending signatures ({pending_count})",
+ "",
+ ]
+ for pending_ref in pending_signatures:
+ lines.append(f"- `{pending_ref}`")
+ else:
+ lines += [
+ "",
+ (
+ "✅ All pipes are concretely implemented — no `PipeSignature` placeholders remain. "
+ "Strict validation will pass; this method is runnable."
+ ),
+ ]
+
+ graph_files = result.get("graph_files")
+ if isinstance(graph_files, dict):
+ lines += ["", "## Graph files", ""]
+ for graph_label, graph_path in cast("dict[str, Any]", graph_files).items():
+ lines.append(f"- **{graph_label}:** `{graph_path}`")
+
+ if "graphspec" in result:
+ lines += ["", "_A GraphSpec (structured graph JSON) is included — use `--format json` to retrieve it._"]
+
+ return "\n".join(lines)
+
+
+def render_invalid_validation_markdown(report: dict[str, Any]) -> str:
+ """Render an invalid validation verdict (the InvalidReport arm) as agent-readable markdown.
+
+ Faithfully renders the structured ``validation_errors`` the hosted ``/validate``
+ InvalidReport carries — the same typed items the agent CLI emits — as a
+ ``# Validation failed`` heading, the summary ``message``, then a numbered
+ ``## Errors`` list with each item's ``category`` + ``message`` and any present
+ locators (pipe / concept / domain / field / source). This is a faithful render
+ of the structured verdict; it does NOT reproduce the agent CLI's generic
+ error-envelope byte-for-byte.
+
+ Args:
+ report: The InvalidReport-shaped dict (``is_valid: False``,
+ ``validation_errors: [...]``, ``message``).
+
+ Returns:
+ A markdown string.
+ """
+ lines: list[str] = ["# Validation failed", ""]
+
+ message = report.get("message")
+ if message:
+ lines += [str(message), ""]
+
+ validation_errors: list[dict[str, Any]] = report.get("validation_errors") or []
+ lines += [f"## Errors ({len(validation_errors)})", ""]
+
+ # Locators rendered in a stable order, only when present (non-None) — mirrors the
+ # structured-info invariant: each item carries the locators it can attribute.
+ locator_labels: list[tuple[str, str]] = [
+ ("pipe", "pipe_code"),
+ ("missing pipe", "missing_pipe_code"),
+ ("concept", "concept_code"),
+ ("missing concept", "missing_concept_code"),
+ ("domain", "domain_code"),
+ ("field", "field_name"),
+ ("path", "field_path"),
+ ("source", "source"),
+ ]
+ for index, item in enumerate(validation_errors, start=1):
+ lines.append(f"{index}. **{item.get('category')}** — {item.get('message')}")
+ for label, key in locator_labels:
+ value = item.get(key)
+ if value:
+ lines.append(f" - {label}: `{value}`")
+
+ return "\n".join(lines)
diff --git a/pipelex/tools/misc/string_utils.py b/pipelex/tools/misc/string_utils.py
index 4b42e16ad..bfcd5425d 100644
--- a/pipelex/tools/misc/string_utils.py
+++ b/pipelex/tools/misc/string_utils.py
@@ -332,6 +332,65 @@ def get_root_from_dotted_path(dotted_path: str) -> str:
return dotted_path.split(".", 1)[0]
+def pluralize(count: int, *, singular: str, plural: str | None = None) -> str:
+ """Pick the singular or plural form of a word based on a count.
+
+ Returns ``singular`` when ``count == 1``, otherwise ``plural`` (defaulting to
+ ``singular`` + "s"). Works for nouns ("pipe"/"pipes") and verbs ("is"/"are").
+
+ Args:
+ count: The quantity that drives the agreement.
+ singular: The form to use when ``count == 1``.
+ plural: The form to use otherwise; defaults to ``singular`` + "s" for
+ regular nouns. Pass it for irregular plurals or verbs.
+
+ Returns:
+ The grammatically-agreed form of the word.
+
+ Examples:
+ >>> pluralize(1, singular="pipe")
+ 'pipe'
+ >>> pluralize(2, singular="pipe")
+ 'pipes'
+ >>> pluralize(0, singular="pipe")
+ 'pipes'
+ >>> pluralize(1, singular="is", plural="are")
+ 'is'
+ >>> pluralize(3, singular="entry", plural="entries")
+ 'entries'
+
+ """
+ if count == 1:
+ return singular
+ return plural or f"{singular}s"
+
+
+def count_with_noun(count: int, *, singular: str, plural: str | None = None) -> str:
+ """Format a count together with its grammatically-agreed noun.
+
+ Produces "1 pipe" / "2 pipes" so messages stay clean instead of resorting to
+ the "(s)" shorthand. Delegates the noun agreement to :func:`pluralize`.
+
+ Args:
+ count: The quantity to render and to drive the noun's agreement.
+ singular: The singular form of the noun.
+ plural: The plural form; defaults to ``singular`` + "s".
+
+ Returns:
+ The count followed by a space and the agreed noun form.
+
+ Examples:
+ >>> count_with_noun(1, singular="pipe")
+ '1 pipe'
+ >>> count_with_noun(2, singular="pipe")
+ '2 pipes'
+ >>> count_with_noun(0, singular="entry", plural="entries")
+ '0 entries'
+
+ """
+ return f"{count} {pluralize(count, singular=singular, plural=plural)}"
+
+
def matches_wildcard_pattern(text: str, *, pattern: str) -> bool:
"""Check if a text matches a wildcard pattern.
diff --git a/pyproject.toml b/pyproject.toml
index aec324077..7d663aad5 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -30,7 +30,7 @@ dependencies = [
"json2html>=1.3.0",
"kajson==0.7.0",
"markdown>=3.6",
- "mthds>=0.4.1",
+ "mthds>=0.5.0",
"networkx>=3.4.2",
"openai>=2.0.0",
"opentelemetry-api",
diff --git a/tests/integration/pipelex/pipeline/test_validate_bundle_structured_errors.py b/tests/integration/pipelex/pipeline/test_validate_bundle_structured_errors.py
new file mode 100644
index 000000000..64479dec0
--- /dev/null
+++ b/tests/integration/pipelex/pipeline/test_validate_bundle_structured_errors.py
@@ -0,0 +1,189 @@
+"""Pin: wiring/concept/type failures surface as STRUCTURED, categorized validation_errors[].
+
+Three failure modes used to collapse to a bare message-only residual (no ``error_type``,
+no locator). They now carry a category + ``error_type`` + identity locators, built by the
+one shared builder and projected onto ``ErrorReport.validation_errors`` (the same items the
+agent CLI emits and the hosted ``InvalidReport`` carries):
+
+- pipe-owned unresolved concept reference (a pipe input/output) → ``pipe_validation`` /
+ ``unresolved_concept`` (with the referencing pipe, the missing concept, and the field).
+- concept-owned unresolved concept reference (a concept's ``refines`` or a structure field's
+ ``concept_ref``) → ``blueprint_validation`` / ``unresolved_concept`` (with the owning concept),
+ so a concept-level failure is never bucketed as a phantom pipe error with a null pipe_code.
+- undefined pipe dependency → ``pipe_validation`` / ``unresolved_pipe_dependency`` (with the
+ referencing pipe in ``pipe_code`` and the missing dependency in ``missing_pipe_code``).
+- unknown pipe ``type`` (a pydantic discriminated-union tag failure) → categorized
+ ``blueprint_validation`` / ``unknown_pipe_type`` with the pipe locator.
+
+Also pins the categorizer fix that made the ``pipe_code`` locator actually populate on
+categorized pipe-blueprint items (the loc key is ``pipe``, not ``pipes``).
+"""
+
+from collections.abc import Callable
+
+import pytest
+
+from pipelex.base_exceptions import ValidationErrorCategory, ValidationErrorItem
+from pipelex.core.pipes.exceptions import PipeValidationErrorType
+from pipelex.pipeline.exceptions import ValidateBundleError
+from pipelex.pipeline.validate_bundle import validate_bundle
+
+_UNKNOWN_CONCEPT_MTHDS = """
+domain = "structured_unknown_concept"
+main_pipe = "make_thing"
+
+[pipe.make_thing]
+type = "PipeLLM"
+description = "Produce an undeclared concept."
+inputs = { topic = "Text" }
+output = "NonExistentConcept"
+prompt = "Write a paragraph about $topic"
+"""
+
+_MISSING_DEPENDENCY_MTHDS = """
+domain = "structured_missing_dep"
+main_pipe = "do_two_steps"
+
+[pipe.do_two_steps]
+type = "PipeSequence"
+description = "Two-step sequence whose second step does not exist."
+inputs = { text = "Text" }
+output = "Text"
+steps = [
+ { pipe = "first_step", result = "interim" },
+ { pipe = "second_step_missing", result = "final" },
+]
+
+[pipe.first_step]
+type = "PipeLLM"
+description = "Echo the text."
+inputs = { text = "Text" }
+output = "Text"
+prompt = "Repeat verbatim: $text"
+"""
+
+_UNKNOWN_PIPE_TYPE_MTHDS = """
+domain = "structured_bad_pipe_type"
+main_pipe = "mystery"
+
+[pipe.mystery]
+type = "PipeWizardry"
+description = "Not a real pipe type."
+inputs = { text = "Text" }
+output = "Text"
+prompt = "Do something magical with $text"
+"""
+
+_PROMPT_INPUT_MISMATCH_MTHDS = """
+domain = "structured_prompt_mismatch"
+main_pipe = "greet_person"
+
+[pipe.greet_person]
+type = "PipeLLM"
+description = "Greeting whose prompt uses an undeclared variable."
+inputs = { name = "Text" }
+output = "Text"
+prompt = "Write a greeting for $name who lives in $city."
+"""
+
+_CONCEPT_OWNED_UNRESOLVED_MTHDS = """
+domain = "structured_concept_owned"
+main_pipe = "use_wrapper"
+
+[concept]
+Wrapper = { description = "Wraps a missing base concept.", refines = "MissingBase" }
+
+[pipe.use_wrapper]
+type = "PipeLLM"
+description = "Produce a wrapper."
+inputs = { text = "Text" }
+output = "Wrapper"
+prompt = "Describe $text"
+"""
+
+
+async def _validation_errors_for(mthds_contents: str) -> list[ValidationErrorItem]:
+ """Validate one invalid bundle and return its structured validation_errors[]."""
+ with pytest.raises(ValidateBundleError) as exc_info:
+ await validate_bundle(mthds_contents=[mthds_contents])
+ report = exc_info.value.to_error_report()
+ items = report.validation_errors
+ assert items, "Every invalid verdict must carry a non-empty validation_errors[]"
+ return items
+
+
[email protected](loop_scope="class")
+class TestValidateBundleStructuredErrors:
+ async def test_unresolved_concept_is_a_categorized_pipe_validation_item(
+ self,
+ load_empty_library: Callable[[], str],
+ ) -> None:
+ load_empty_library()
+ items = await _validation_errors_for(_UNKNOWN_CONCEPT_MTHDS)
+ concept_items = [item for item in items if item.error_type == PipeValidationErrorType.UNRESOLVED_CONCEPT]
+ assert concept_items, f"Expected an unresolved_concept item, got {[(i.category, i.error_type) for i in items]}"
+ item = concept_items[0]
+ assert item.category == ValidationErrorCategory.PIPE_VALIDATION
+ assert item.pipe_code == "make_thing"
+ assert item.concept_code == "NonExistentConcept"
+ assert item.field_name == "output"
+ assert item.domain_code == "structured_unknown_concept"
+
+ async def test_concept_owned_unresolved_concept_is_a_categorized_blueprint_item(
+ self,
+ load_empty_library: Callable[[], str],
+ ) -> None:
+ """A concept-owned ref (here a `refines`) surfaces as a blueprint_validation item on the OWNING
+ concept, not as a phantom pipe_validation item with a null pipe_code.
+ """
+ load_empty_library()
+ items = await _validation_errors_for(_CONCEPT_OWNED_UNRESOLVED_MTHDS)
+ concept_items = [item for item in items if item.error_type == PipeValidationErrorType.UNRESOLVED_CONCEPT]
+ assert concept_items, f"Expected an unresolved_concept item, got {[(i.category, i.error_type) for i in items]}"
+ item = concept_items[0]
+ assert item.category == ValidationErrorCategory.BLUEPRINT_VALIDATION
+ assert item.pipe_code is None
+ assert item.concept_code == "Wrapper"
+ assert item.domain_code == "structured_concept_owned"
+ assert "MissingBase" in item.message
+
+ async def test_unresolved_pipe_dependency_is_a_categorized_pipe_validation_item(
+ self,
+ load_empty_library: Callable[[], str],
+ ) -> None:
+ load_empty_library()
+ items = await _validation_errors_for(_MISSING_DEPENDENCY_MTHDS)
+ dependency_items = [item for item in items if item.error_type == PipeValidationErrorType.UNRESOLVED_PIPE_DEPENDENCY]
+ assert dependency_items, f"Expected an unresolved_pipe_dependency item, got {[(i.category, i.error_type) for i in items]}"
+ item = dependency_items[0]
+ assert item.category == ValidationErrorCategory.PIPE_VALIDATION
+ assert item.pipe_code == "do_two_steps"
+ assert item.missing_pipe_code == "second_step_missing"
+ assert "second_step_missing" in item.message
+
+ async def test_unknown_pipe_type_is_a_categorized_blueprint_item(
+ self,
+ load_empty_library: Callable[[], str],
+ ) -> None:
+ load_empty_library()
+ items = await _validation_errors_for(_UNKNOWN_PIPE_TYPE_MTHDS)
+ type_items = [item for item in items if item.error_type == PipeValidationErrorType.UNKNOWN_PIPE_TYPE]
+ assert type_items, f"Expected an unknown_pipe_type item, got {[(i.category, i.error_type) for i in items]}"
+ item = type_items[0]
+ assert item.category == ValidationErrorCategory.BLUEPRINT_VALIDATION
+ assert item.pipe_code == "mystery"
+ assert item.domain_code == "structured_bad_pipe_type"
+
+ async def test_missing_input_variable_now_carries_the_pipe_code_locator(
+ self,
+ load_empty_library: Callable[[], str],
+ ) -> None:
+ """Regression for the categorizer loc-key fix: pipe_code now populates on pipe blueprint items."""
+ load_empty_library()
+ items = await _validation_errors_for(_PROMPT_INPUT_MISMATCH_MTHDS)
+ missing_items = [item for item in items if item.error_type == PipeValidationErrorType.MISSING_INPUT_VARIABLE]
+ assert missing_items, f"Expected a missing_input_variable item, got {[(i.category, i.error_type) for i in items]}"
+ item = missing_items[0]
+ assert item.category == ValidationErrorCategory.BLUEPRINT_VALIDATION
+ assert item.pipe_code == "greet_person"
+ assert item.variable_names == ["city"]
diff --git a/tests/unit/pipelex/cli/agent_cli/test_validate_output_helpers.py b/tests/unit/pipelex/cli/agent_cli/test_validate_output_helpers.py
deleted file mode 100644
index 8cdd943b1..000000000
--- a/tests/unit/pipelex/cli/agent_cli/test_validate_output_helpers.py
+++ /dev/null
@@ -1,73 +0,0 @@
-"""Unit tests for the agent CLI validate markdown renderer (`format_validate_markdown`)."""
-
-from __future__ import annotations
-
-from typing import Any
-
-from pipelex.cli.agent_cli.commands.validate._output_helpers import format_validate_markdown # noqa: PLC2701
-
-
-class TestFormatValidateMarkdown:
- def test_renders_pending_signatures_section_with_not_runnable_verdict(self):
- """A populated pending_signatures list renders the verbatim section heading, an explicit
- "NOT yet runnable" verdict, and each ref as a bullet.
- """
- result: dict[str, Any] = {
- "success": True,
- "bundle_path": "/fake/method.mthds",
- "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
- "total_pipes": 1,
- "pending_signatures": ["research.find_key_findings", "research.rank_findings"],
- "is_runnable": False,
- }
-
- markdown = format_validate_markdown(result)
-
- assert markdown.startswith("# Validation passed")
- # Heading kept verbatim — a downstream plugin reads this exact string.
- assert "## Pending signatures (2)" in markdown
- # Explicit negative verdict, with the count, immediately above the list.
- assert "⚠️ This method is NOT yet runnable" in markdown
- assert "2 pipe(s) are still `PipeSignature` placeholders" in markdown
- assert "- `research.find_key_findings`" in markdown
- assert "- `research.rank_findings`" in markdown
- # The verdict must precede the verbatim heading — consumers expect the note directly above it.
- assert markdown.index("⚠️ This method is NOT yet runnable") < markdown.index("## Pending signatures")
-
- def test_renders_runnable_verdict_for_complete_bundle(self):
- """A present-but-empty pending_signatures (complete bundle) renders an explicit runnable
- verdict and no "Pending signatures" section.
- """
- result: dict[str, Any] = {
- "success": True,
- "bundle_path": "/fake/method.mthds",
- "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
- "total_pipes": 1,
- "pending_signatures": [],
- "is_runnable": True,
- }
-
- markdown = format_validate_markdown(result)
-
- assert markdown.startswith("# Validation passed")
- assert "✅ All pipes are concretely implemented" in markdown
- assert "this method is runnable." in markdown
- assert "Pending signatures" not in markdown
-
- def test_omits_runnability_verdict_when_key_absent(self):
- """The `validate all` / `validate pipe` shape omits the `pending_signatures` key entirely —
- so no runnability verdict (positive or negative) leaks onto those non-bundle surfaces.
- """
- without_key: dict[str, Any] = {
- "success": True,
- "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
- "total_pipes": 1,
- }
-
- markdown = format_validate_markdown(without_key)
-
- assert markdown.startswith("# Validation passed")
- assert "Pending signatures" not in markdown
- assert "runnable" not in markdown
- assert "✅" not in markdown
- assert "⚠️" not in markdown
diff --git a/tests/unit/pipelex/cli/test_agent_validate_cmd.py b/tests/unit/pipelex/cli/test_agent_validate_cmd.py
index 306434f45..fa48e920f 100644
--- a/tests/unit/pipelex/cli/test_agent_validate_cmd.py
+++ b/tests/unit/pipelex/cli/test_agent_validate_cmd.py
@@ -75,7 +75,8 @@ class TestValidateBundleCmd:
output_format=CliOutputFormat.JSON,
)
- assert exc_info.value.exit_code == 1
+ # Graph generation failure is a no-verdict condition → exit 2 (validate 0/1/2 policy).
+ assert exc_info.value.exit_code == 2
stderr_text = capsys.readouterr().err
# Parse all JSON objects from stderr — there should be exactly one
diff --git a/tests/unit/pipelex/cli/test_validate_exit_codes.py b/tests/unit/pipelex/cli/test_validate_exit_codes.py
new file mode 100644
index 000000000..30356a7e4
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_validate_exit_codes.py
@@ -0,0 +1,222 @@
+"""The validate 0/1/2 exit-code policy on both CLIs (bare ``pipelex validate`` + ``pipelex-agent validate``).
+
+Policy: exit 0 = valid; exit 1 = a produced NEGATIVE VERDICT (invalid bundle, or
+valid-but-not-runnable without ``--allow-signatures``); exit 2 = NO VERDICT (bad
+args, unresolvable target, file-not-found, unexpected). See
+``docs/specs/pipelex-validation-api.md`` (CLI exit-code policy) and
+``docs/specs/pipelex-mthds-protocol.md`` (Agent-CLI validate envelope).
+
+The pre-boot argument/target-resolution cases are pure (they raise before any
+Pipelex boot); the negative-verdict / no-verdict / signature-gate cases mock the
+validate core (mirroring ``test_agent_validate_cmd.py``). The real end-to-end
+exit codes against the binary are pinned by the conformance suite.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+
+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat
+from pipelex.cli.agent_cli.commands.validate.bundle_cmd import validate_bundle_cmd as agent_validate_bundle_cmd
+from pipelex.cli.agent_cli.commands.validate.pipe_cmd import validate_pipe_cmd as agent_validate_pipe_cmd
+from pipelex.cli.commands.validate.bundle_cmd import validate_bundle_cmd as bare_validate_bundle_cmd
+from pipelex.cli.commands.validate.pipe_cmd import validate_pipe_cmd as bare_validate_pipe_cmd
+from pipelex.pipe_run.exceptions import DryRunError
+from pipelex.pipeline.exceptions import ValidateBundleError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+AGENT_BUNDLE_MODULE = "pipelex.cli.agent_cli.commands.validate.bundle_cmd"
+AGENT_PIPE_MODULE = "pipelex.cli.agent_cli.commands.validate.pipe_cmd"
+
+
+def _make_dir(tmp_path: Path, name: str, mthds_files: list[str]) -> Path:
+ """Create a directory containing the given .mthds filenames (empty content)."""
+ directory = tmp_path / name
+ directory.mkdir()
+ for filename in mthds_files:
+ (directory / filename).write_text('[domain]\ncode = "test"\n')
+ return directory
+
+
+class TestValidateExitCodes:
+ """The validate 0/1/2 exit-code policy on both CLIs.
+
+ ``bare_*`` cover the bare ``pipelex validate`` group; ``agent_*`` cover
+ ``pipelex-agent validate``. The pre-boot bad-args / target-resolution cases
+ exit 2 before any Pipelex boot; the verdict cases mock the validate core.
+ """
+
+ # --- bare CLI: bad args / unresolvable targets exit 2 (no verdict), pre-boot ---
+
+ def test_bare_bundle_directory_without_mthds_exits_2(self, tmp_path: Path) -> None:
+ empty_dir = _make_dir(tmp_path, "empty", [])
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_bundle_cmd(path=str(empty_dir))
+ assert exc_info.value.exit_code == 2
+
+ def test_bare_bundle_directory_with_multiple_mthds_and_no_default_exits_2(self, tmp_path: Path) -> None:
+ multi_dir = _make_dir(tmp_path, "multi", ["a.mthds", "b.mthds"])
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_bundle_cmd(path=str(multi_dir))
+ assert exc_info.value.exit_code == 2
+
+ def test_bare_bundle_path_that_is_neither_file_nor_dir_exits_2(self, tmp_path: Path) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_bundle_cmd(path=str(tmp_path / "nope"))
+ assert exc_info.value.exit_code == 2
+
+ def test_bare_pipe_all_combined_with_code_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_pipe_cmd(pipe_code="some_pipe", validate_all=True)
+ assert exc_info.value.exit_code == 2
+
+ def test_bare_pipe_without_code_or_all_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_pipe_cmd(pipe_code=None)
+ assert exc_info.value.exit_code == 2
+
+ def test_bare_pipe_code_that_looks_like_a_path_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ bare_validate_pipe_cmd(pipe_code="my_bundle.mthds")
+ assert exc_info.value.exit_code == 2
+
+ # --- agent CLI: bad args / unresolvable targets exit 2 (no verdict), pre-boot ---
+
+ def test_agent_bundle_directory_without_mthds_exits_2(self, tmp_path: Path) -> None:
+ empty_dir = _make_dir(tmp_path, "empty", [])
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(empty_dir), output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_agent_bundle_directory_with_multiple_mthds_exits_2(self, tmp_path: Path) -> None:
+ multi_dir = _make_dir(tmp_path, "multi", ["a.mthds", "b.mthds"])
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(multi_dir), output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_agent_bundle_path_that_is_neither_file_nor_dir_exits_2(self, tmp_path: Path) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(tmp_path / "nope"), output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_agent_pipe_all_combined_with_code_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_pipe_cmd(pipe_code="some_pipe", validate_all=True, output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_agent_pipe_without_code_or_all_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_pipe_cmd(pipe_code=None, output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_agent_pipe_code_that_looks_like_a_path_exits_2(self) -> None:
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_pipe_cmd(pipe_code="my_bundle.mthds", output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ # --- agent CLI bundle: negative verdict (1), no verdict (2), signature gate, valid (0) ---
+
+ def _bundle_file(self, tmp_path: Path) -> Path:
+ mthds_file = tmp_path / "bundle.mthds"
+ mthds_file.write_text('[bundle]\nmain_pipe = "my_pipe"\n[domain]\ncode = "test"\n')
+ return mthds_file
+
+ def _patch_boot(self, mocker: MockerFixture) -> None:
+ mocker.patch(f"{AGENT_BUNDLE_MODULE}.make_pipelex_for_agent_cli")
+ mocker.patch(f"{AGENT_BUNDLE_MODULE}.Pipelex.teardown_if_needed")
+
+ def test_invalid_bundle_is_negative_verdict_exit_1(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ self._patch_boot(mocker)
+ mocker.patch(
+ f"{AGENT_BUNDLE_MODULE}.validate_bundle_core",
+ new=mocker.AsyncMock(side_effect=ValidateBundleError("Bundle is invalid", dry_run_error_message="boom")),
+ )
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(self._bundle_file(tmp_path)), output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 1
+
+ def test_bundle_file_not_found_is_no_verdict_exit_2(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ self._patch_boot(mocker)
+ mocker.patch(
+ f"{AGENT_BUNDLE_MODULE}.validate_bundle_core",
+ new=mocker.AsyncMock(side_effect=FileNotFoundError("missing")),
+ )
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(self._bundle_file(tmp_path)), output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 2
+
+ def test_not_runnable_without_allow_signatures_is_negative_verdict_exit_1(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ self._patch_boot(mocker)
+ not_runnable: dict[str, Any] = {
+ "success": True,
+ "is_valid": True,
+ "bundle_path": str(self._bundle_file(tmp_path)),
+ "validated_pipes": [{"pipe_ref": "test.my_pipe", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": ["test.todo"],
+ "is_runnable": False,
+ }
+ mocker.patch(f"{AGENT_BUNDLE_MODULE}.validate_bundle_core", new=mocker.AsyncMock(return_value=not_runnable))
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_bundle_cmd(path=str(self._bundle_file(tmp_path)), output_format=CliOutputFormat.JSON, allow_signatures=False)
+ assert exc_info.value.exit_code == 1
+
+ def test_not_runnable_with_allow_signatures_exits_0(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ self._patch_boot(mocker)
+ not_runnable: dict[str, Any] = {
+ "success": True,
+ "is_valid": True,
+ "bundle_path": str(self._bundle_file(tmp_path)),
+ "validated_pipes": [{"pipe_ref": "test.my_pipe", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": ["test.todo"],
+ "is_runnable": False,
+ }
+ mocker.patch(f"{AGENT_BUNDLE_MODULE}.validate_bundle_core", new=mocker.AsyncMock(return_value=not_runnable))
+ # --allow-signatures tolerates the placeholders: the command returns normally (exit 0), no typer.Exit.
+ agent_validate_bundle_cmd(path=str(self._bundle_file(tmp_path)), output_format=CliOutputFormat.JSON, allow_signatures=True)
+
+ def test_valid_runnable_bundle_exits_0(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ self._patch_boot(mocker)
+ runnable: dict[str, Any] = {
+ "success": True,
+ "is_valid": True,
+ "bundle_path": str(self._bundle_file(tmp_path)),
+ "validated_pipes": [{"pipe_ref": "test.my_pipe", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": [],
+ "is_runnable": True,
+ }
+ mocker.patch(f"{AGENT_BUNDLE_MODULE}.validate_bundle_core", new=mocker.AsyncMock(return_value=runnable))
+ # Valid and runnable: returns normally (exit 0), no typer.Exit raised.
+ agent_validate_bundle_cmd(path=str(self._bundle_file(tmp_path)), output_format=CliOutputFormat.JSON)
+
+ def _patch_pipe_boot(self, mocker: MockerFixture) -> None:
+ mocker.patch(f"{AGENT_PIPE_MODULE}.make_pipelex_for_agent_cli")
+ mocker.patch(f"{AGENT_PIPE_MODULE}.Pipelex.teardown_if_needed")
+
+ def test_agent_validate_all_dry_run_failure_is_negative_verdict_exit_1(self, mocker: MockerFixture) -> None:
+ # `validate --all` sweeps via validate_all_core → validate_current_library, which raises
+ # DryRunError directly. A dry-run failure is a produced negative verdict → exit 1, NOT the
+ # catch-all's no-verdict 2.
+ self._patch_pipe_boot(mocker)
+ mocker.patch(f"{AGENT_PIPE_MODULE}.validate_all_core", new=mocker.AsyncMock(side_effect=DryRunError("pipe dry-run failed")))
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_pipe_cmd(pipe_code=None, validate_all=True, output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 1
+
+ def test_agent_validate_pipe_dry_run_failure_is_negative_verdict_exit_1(self, mocker: MockerFixture) -> None:
+ # Single-pipe validate_pipe_core sweeps via validate_pipes, which raises DryRunError directly.
+ self._patch_pipe_boot(mocker)
+ mocker.patch(f"{AGENT_PIPE_MODULE}.resolve_pipe_from_exports", new=mocker.Mock(return_value=[]))
+ mocker.patch(f"{AGENT_PIPE_MODULE}.validate_pipe_core", new=mocker.AsyncMock(side_effect=DryRunError("pipe dry-run failed")))
+ with pytest.raises(typer.Exit) as exc_info:
+ agent_validate_pipe_cmd(pipe_code="my_pipe", output_format=CliOutputFormat.JSON)
+ assert exc_info.value.exit_code == 1
diff --git a/tests/unit/pipelex/pipeline/test_validation_render.py b/tests/unit/pipelex/pipeline/test_validation_render.py
new file mode 100644
index 000000000..b89b6777c
--- /dev/null
+++ b/tests/unit/pipelex/pipeline/test_validation_render.py
@@ -0,0 +1,132 @@
+"""Unit tests for the validate markdown renderers in `pipelex.pipeline.validation_render`."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from pipelex.pipeline.validation_render import format_validate_markdown, render_invalid_validation_markdown
+
+
+class TestValidationRender:
+ def test_renders_pending_signatures_section_with_not_runnable_verdict(self):
+ """A populated pending_signatures list renders the verbatim section heading, an explicit
+ "NOT yet runnable" verdict, and each ref as a bullet.
+ """
+ result: dict[str, Any] = {
+ "success": True,
+ "bundle_path": "/fake/method.mthds",
+ "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": ["research.find_key_findings", "research.rank_findings"],
+ "is_runnable": False,
+ }
+
+ markdown = format_validate_markdown(result)
+
+ assert markdown.startswith("# Validation passed")
+ # Heading kept verbatim — a downstream plugin reads this exact string.
+ assert "## Pending signatures (2)" in markdown
+ # Explicit negative verdict, with the count, immediately above the list.
+ assert "⚠️ This method is NOT yet runnable" in markdown
+ assert "2 pipes are still `PipeSignature` placeholders" in markdown
+ assert "- `research.find_key_findings`" in markdown
+ assert "- `research.rank_findings`" in markdown
+ # The verdict must precede the verbatim heading — consumers expect the note directly above it.
+ assert markdown.index("⚠️ This method is NOT yet runnable") < markdown.index("## Pending signatures")
+
+ def test_renders_single_pending_signature_with_singular_grammar(self):
+ """A single pending signature renders grammatically — "1 pipe is still a `PipeSignature`
+ placeholder" — with no "(s)", singular verb, and the article present.
+ """
+ result: dict[str, Any] = {
+ "success": True,
+ "bundle_path": "/fake/method.mthds",
+ "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": ["research.find_key_findings"],
+ "is_runnable": False,
+ }
+
+ markdown = format_validate_markdown(result)
+
+ # The count line also pluralizes: a single validated pipe is "1 pipe", not "1 pipe(s)".
+ assert "Validated 1 pipe:" in markdown
+ assert "1 pipe is still a `PipeSignature` placeholder" in markdown
+ assert "## Pending signatures (1)" in markdown
+ # No "(s)" shorthand anywhere in the rendered markdown.
+ assert "(s)" not in markdown
+
+ def test_renders_runnable_verdict_for_complete_bundle(self):
+ """A present-but-empty pending_signatures (complete bundle) renders an explicit runnable
+ verdict and no "Pending signatures" section.
+ """
+ result: dict[str, Any] = {
+ "success": True,
+ "bundle_path": "/fake/method.mthds",
+ "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ "pending_signatures": [],
+ "is_runnable": True,
+ }
+
+ markdown = format_validate_markdown(result)
+
+ assert markdown.startswith("# Validation passed")
+ assert "✅ All pipes are concretely implemented" in markdown
+ assert "this method is runnable." in markdown
+ assert "Pending signatures" not in markdown
+
+ def test_omits_runnability_verdict_when_key_absent(self):
+ """The `validate all` / `validate pipe` shape omits the `pending_signatures` key entirely —
+ so no runnability verdict (positive or negative) leaks onto those non-bundle surfaces.
+ """
+ without_key: dict[str, Any] = {
+ "success": True,
+ "validated_pipes": [{"pipe_ref": "research.research_brief", "status": "SUCCESS"}],
+ "total_pipes": 1,
+ }
+
+ markdown = format_validate_markdown(without_key)
+
+ assert markdown.startswith("# Validation passed")
+ assert "Pending signatures" not in markdown
+ assert "runnable" not in markdown
+ assert "✅" not in markdown
+ assert "⚠️" not in markdown
+
+ def test_renders_invalid_report_with_locators_and_dry_run_residual(self):
+ """The invalid-arm renderer surfaces the message, an error count, and each item's
+ category + message with its present locators; a dry_run residual (no locators) renders too.
+ """
+ report: dict[str, Any] = {
+ "is_valid": False,
+ "message": "MTHDS validation found errors",
+ "validation_errors": [
+ {
+ "category": "pipe_validation",
+ "error_type": "PipeValidationError",
+ "message": "Pipe references an unknown concept.",
+ "pipe_code": "summarize",
+ "concept_code": "Contractt",
+ "field_name": "output",
+ "source": "/fake/method.mthds",
+ },
+ {
+ "category": "dry_run",
+ "error_type": "DryRunError",
+ "message": "Dry run failed: boom.",
+ },
+ ],
+ }
+
+ markdown = render_invalid_validation_markdown(report)
+
+ assert markdown.startswith("# Validation failed")
+ assert "MTHDS validation found errors" in markdown
+ assert "## Errors (2)" in markdown
+ assert "1. **pipe_validation** — Pipe references an unknown concept." in markdown
+ assert " - pipe: `summarize`" in markdown
+ assert " - concept: `Contractt`" in markdown
+ assert " - source: `/fake/method.mthds`" in markdown
+ # The dry_run residual has no locators — it renders its category + message, no sub-bullets.
+ assert "2. **dry_run** — Dry run failed: boom." in markdown
diff --git a/tests/unit/pipelex/tools/misc/test_string_utils.py b/tests/unit/pipelex/tools/misc/test_string_utils.py
index 6eaab25a3..a978be73a 100644
--- a/tests/unit/pipelex/tools/misc/test_string_utils.py
+++ b/tests/unit/pipelex/tools/misc/test_string_utils.py
@@ -3,6 +3,7 @@ import pytest
from pipelex.tools.misc.string_utils import (
camel_to_snake_case,
can_inject_text,
+ count_with_noun,
get_root_from_dotted_path,
is_none_or_has_text,
is_not_none_and_has_text,
@@ -13,6 +14,7 @@ from pipelex.tools.misc.string_utils import (
pascal_case_to_kebab,
pascal_case_to_sentence,
pascal_case_to_snake_case,
+ pluralize,
snake_to_capitalize_first_letter,
snake_to_pascal_case,
)
@@ -256,6 +258,39 @@ def test_get_root_from_dotted_path(dotted_path: str, expected: str) -> None:
assert get_root_from_dotted_path(dotted_path) == expected
+ ("count", "singular", "plural", "expected"),
+ [
+ # Regular noun: singular at 1, default "+s" plural otherwise
+ (1, "pipe", None, "pipe"),
+ (2, "pipe", None, "pipes"),
+ (0, "pipe", None, "pipes"),
+ # Explicit irregular plural
+ (1, "entry", "entries", "entry"),
+ (3, "entry", "entries", "entries"),
+ # Verb agreement
+ (1, "is", "are", "is"),
+ (2, "is", "are", "are"),
+ ],
+)
+def test_pluralize(count: int, singular: str, plural: str | None, expected: str) -> None:
+ assert pluralize(count, singular=singular, plural=plural) == expected
+
+
+ ("count", "singular", "plural", "expected"),
+ [
+ (1, "pipe", None, "1 pipe"),
+ (2, "pipe", None, "2 pipes"),
+ (0, "pipe", None, "0 pipes"),
+ (1, "entry", "entries", "1 entry"),
+ (5, "entry", "entries", "5 entries"),
+ ],
+)
+def test_count_with_noun(count: int, singular: str, plural: str | None, expected: str) -> None:
+ assert count_with_noun(count, singular=singular, plural=plural) == expected
+
+
@pytest.mark.parametrize(
("text", "expected"),
[
diff --git a/uv.lock b/uv.lock
index 43c9f4243..f2445978e 100644
--- a/uv.lock
+++ b/uv.lock
@@ -2664,7 +2664,7 @@ wheels = [
[[package]]
name = "mthds"
-version = "0.4.1"
+version = "0.5.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "backports-strenum", marker = "python_full_version < '3.11'" },
@@ -2675,9 +2675,9 @@ dependencies = [
{ name = "tomlkit" },
{ name = "typing-extensions" },
]
-sdist = { url = "https://files.pythonhosted.org/packages/46/fc/7822e39bc733f5f62577ea1d064c7283d84b97493e5372ceb430ce0c1027/mthds-0.4.1.tar.gz", hash = "sha256:1aa43860fef97aeab561473e45552ea619dee7c624bc4acad806dd960bb2fe5f", size = 132336, upload-time = "2026-06-11T21:46:28.332Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/3a/6b/5d706bade53e93466fbe1816d3046b7924c67bfec9859544cbedf21ea4c5/mthds-0.5.0.tar.gz", hash = "sha256:143b4405590d4f78713c442ad55553542d630c374099e9ee02b96823b2a2c426", size = 142082, upload-time = "2026-06-18T20:45:24.601Z" }
wheels = [
- { url = "https://files.pythonhosted.org/packages/5a/92/8d6521c63e71fc4b782d7a875b27ad5b61cf1af718eb3321a3522aa33391/mthds-0.4.1-py3-none-any.whl", hash = "sha256:fdbf4ec25f4139b620cf5771fb8dc9fa27c2c249c161ca96afac389f2cfc6d4f", size = 61176, upload-time = "2026-06-11T21:46:26.978Z" },
+ { url = "https://files.pythonhosted.org/packages/ef/47/0af23c995b6859b8809d077185d97de1be93c50f56bfe980daaf7cd9fc2d/mthds-0.5.0-py3-none-any.whl", hash = "sha256:a41a7b5986b65f41b0465372fc4f9180800ad212ed67830d8bb8d6dd457cd220", size = 64413, upload-time = "2026-06-18T20:45:23.277Z" },
]
[[package]]
@@ -3864,7 +3864,7 @@ requires-dist = [
{ name = "mkdocs-meta-manager", marker = "extra == 'docs'", specifier = ">=1.1.0" },
{ name = "mkdocs-redirects", marker = "extra == 'docs'", specifier = "==1.2.2" },
{ name = "moto", extras = ["dynamodb", "s3"], marker = "extra == 'dev'", specifier = ">=5.0.0" },
- { name = "mthds", specifier = ">=0.4.1" },
+ { name = "mthds", specifier = ">=0.5.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = "==1.19.1" },
{ name = "networkx", specifier = ">=3.4.2" },
{ name = "openai", specifier = ">=2.0.0" },
diff --git a/wip/README.md b/wip/README.md
index 5d97b25ff..f3d66bac8 100644
--- a/wip/README.md
+++ b/wip/README.md
@@ -26,6 +26,7 @@ This folder holds the **active** planning, design, and current-state docs for th
- **[stuffs-as-nodespec.md](stuffs-as-nodespec.md)** — future graph-model rework to make stuffs first-class `NodeSpec` instances. A starter, not a worked plan.
- **[blueprint-elaboration-directives.md](blueprint-elaboration-directives.md)** — extending the build-time blueprint-elaboration system: first-class synthetic-pipe handling, a directive plugin registry, and supporting tooling.
- **[doctor-output-ansi-under-force-color.md](doctor-output-ansi-under-force-color.md)** — deferred decision: `pipelex doctor` emits ANSI when captured under `FORCE_COLOR` (an env artifact, not a pipelex defect — it's plain when piped without it). Records the evidence and the one optional follow-up (a `--plain`/`--no-color` flag for diagnostics). Low priority. Surfaced by pipelex-worker PR #23.
+- **[structured-validation-errors-deferred-findings.md](structured-validation-errors-deferred-findings.md)** — lower-severity code-review follow-ups on the structured wiring/concept/type `validation_errors[]` work (`feature/Validate-api-render-format`): the missing-`type` (`union_tag_not_found`) categorization gap, dependency-validation fail-fast vs. concept-ref accumulation, qualified same-domain `concept_code`, the source-less `unresolved_pipe_dependency` item, a duplicate-message cleanup, and the `concept_code`-meaning trade-off baked in by the concept-owned fix. All low priority; each a contained pickup.
- **[pathlib-refactor.workflow.js](pathlib-refactor.workflow.js)** — a runnable multi-agent workflow that refactors filesystem path handling to `pathlib.Path` across `pipelex/` (intended for the `refactor/Paths` branch). Tooling, not a doc.
## Suggested priorities
diff --git a/wip/structured-validation-errors-deferred-findings.md b/wip/structured-validation-errors-deferred-findings.md
new file mode 100644
index 000000000..9d6263d8b
--- /dev/null
+++ b/wip/structured-validation-errors-deferred-findings.md
@@ -0,0 +1,52 @@
+# Deferred: structured validation-errors follow-ups
+
+Code-review follow-ups on the structured wiring/concept/type validation-errors work (`feature/Validate-api-render-format`). The two consumer-facing findings from that review (concept-owned refs mislabeled as `pipe_validation`; the missing dependency only in the message) were **fixed** in that change. The items below were judged lower-severity and **deferred** here rather than fixed inline — each is a contained, independent pickup. None is a silent correctness regression; they are coverage gaps, asymmetries, and one cleanup.
+
+The surface: `validate_bundle` → `_translate_to_validate_bundle_error` (`pipelex/pipeline/validate_bundle.py`) → `build_validation_error_items` (`pipelex/pipeline/validation_errors.py`) → `ErrorReport.validation_errors[]` (`ValidationErrorItem` in `pipelex/base_exceptions.py`). The three categorizing sites are `categorize_blueprint_validation_error` (`pipelex/core/interpreter/validation_error_categorizer.py`), `validate_concept_references_in_blueprints` (`pipelex/libraries/concept_reference_validation.py`), and `Library.validate_pipe_library_with_libraries` (`pipelex/libraries/library.py`).
+
+## 1. Missing-`type` pipe not categorized (`union_tag_not_found`)
+
+`categorize_blueprint_validation_error` only categorizes `error["type"] == "union_tag_invalid"` as `UNKNOWN_PIPE_TYPE` (the case where a pipe declares a `type` matching no known operator/controller). A pipe block that **omits `type` entirely** raises pydantic `union_tag_not_found`, which the branch does not match — so it falls through to the uncategorized `log.warning(...)` residual with no `error_type` / `pipe_code`.
+
+This is asymmetric with the present-but-unknown-`type` case the feature fixes: both are "bad/absent pipe type" from the author's view, but only one is categorized.
+
+**Shape when picked up:** match both `union_tag_invalid` and `union_tag_not_found` in that branch (the loc is still `("pipe", <code>)`, so `pipe_code` populates the same way), and add a fixture with a `[pipe.foo]` block that omits `type`. Verify pydantic's exact error code for the missing-discriminator case first.
+
+## 2. Dependency validation fails fast; only the first missing sub-pipe is reported
+
+In `Library.validate_pipe_library_with_libraries`, the `raise LibraryLoadingError(...)` for an unresolved dependency sits **inside** the `for sub_pipe_code in pipe.pipe_dependencies()` loop, so a controller with several missing dependencies surfaces only **one** `unresolved_pipe_dependency` item. This contrasts with `validate_concept_references_in_blueprints`, which deliberately accumulates every unresolved reference and reports them together ("the author sees all of them at once").
+
+**Shape when picked up:** accumulate `PipesAndConceptValidationErrorData` items across the dependency loop (and likely across pipes) and raise once at the end, mirroring the concept-ref accumulation. Note this is a behavioral change to a fail-fast path that predates the feature; scope the blast radius (other callers of `validate_pipe_library_with_libraries` rely on it raising on the first failure) before changing it.
+
+## 3. Qualified same-domain ref yields a qualified `concept_code`
+
+In `validate_concept_references_in_blueprints`, the pipe-owned `unresolved_concept` item sets `concept_code=concept_ref_or_code` — the **raw** ref. For a same-domain ref that the author happened to qualify (e.g. `output = "mydomain.Foo"` in domain `mydomain`), `ref.is_external_to("mydomain")` is `False`, so it reaches the unresolved branch and `concept_code` becomes the domain-qualified string `"mydomain.Foo"` rather than the bare `Foo`. The categorizer's concept path and the bare-ref test (`concept_code == "NonExistentConcept"`) assume the bare code, so a consumer keying on a bare `concept_code` mismatches when the author qualified the ref.
+
+**Shape when picked up:** decide the canonical form for `concept_code` (bare local code vs. as-written) and normalize at the construction site — likely `ref.local_code` for the bare form — then add a fixture with a qualified same-domain output ref.
+
+## 4. `unresolved_pipe_dependency` carries no `source`
+
+The `unresolved_pipe_dependency` item omits `source`, while the sibling `unresolved_concept` and `unknown_pipe_type` items carry it. Root cause: at library-validation time `library.py` builds the item from the `pipe` object, and `PipeAbstract` has no `source` field — so there is no per-pipe declaring-file path to attach. A consumer mapping `validation_errors[]` to declaring files (the vscode cross-file diagnostics path) can attribute the concept/type failures but not the dependency one.
+
+This is a **model limitation**, not a regression. Fixing it means threading the declaring source onto the pipe model (or looking it up by domain at validation time) — non-trivial.
+
+Related but distinct: [`validate-parse-level-source-attribution.md`](validate-parse-level-source-attribution.md) covers the *parse-level* (malformed-TOML) source-less residual. Both are about `source` on `validation_errors[]`, but that one is in the parse comprehension and this one is in library validation — they would be fixed in different places.
+
+## 5. (cleanup) Per-reference failure sentence is built twice
+
+In `validate_concept_references_in_blueprints`, each unresolved reference builds two near-identical sentences in the same iteration: the `undeclared` aggregate-message entry (`'{ref}' in {context} is not declared in domain '{domain}' (source: '{source}')`) and the per-item `item_message` (`Concept '{ref}' in {context} is not declared in domain '{domain}' and is not native.`). A future wording change must touch both or the aggregate `ConceptLibraryError` message and the structured item silently diverge.
+
+**Shape when picked up:** derive one from the other (or from a single small formatter), so there is a single phrasing of "this reference does not resolve."
+
+## Known trade-off baked in by the concept-owned fix (revisit only if it bites)
+
+The concept-owned-ref fix made `concept_code` mean different things across categories for `error_type == unresolved_concept`:
+
+- pipe-owned (`pipe_validation`): `pipe_code` = owning pipe, `concept_code` = **missing** concept.
+- concept-owned (`blueprint_validation`): `concept_code` = **owning** concept (consistent with how every other `blueprint_validation` concept error is categorized); the missing reference is named in the `message`.
+
+A consumer disambiguates via `category`, and both branches carry comments. The clean-but-larger alternative would be distinct "owner" and "missing/referenced" locator fields on the wire item (and on `PipesAndConceptValidationErrorData` / `PipelexBundleBlueprintValidationErrorData`) so `concept_code` need not double as both. Deferred as a wire-model design question, not worth the churn unless a consumer needs the owner concept structurally in the pipe-owned case (or the missing concept structurally in the concept-owned case).
+
+## Cross-repo note
+
+Any change here that alters the `validation_errors[]` wire shape (new locator fields, normalized `concept_code`) must be mirrored in `conformance/conformance/validation_contract.py` + `conformance/conformance/assertions/json_output.py`, per the spec/conformance sync rule, and the conformance suite run on both arms.