Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix: dynamic concepts get no static prompt classification in the template analyzers (#1179)

Louis Choquel789726a072e3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8ae4ea978..0b9efa783 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@
- **An unknown or ambiguous entry `pipe_code` is reported as the caller's mistake, not a server fault**: the two entry-shaped lookups — the ones a CLI argument and an API request field land in — raised the bare `PipeNotFoundError` / `PipeLibraryError`, which declare no `error_domain`, so `ErrorReport.http_status` fell through to 500. A caller typo in the `pipe_code` of a run request therefore came back as an internal error, logged with a traceback and looking retryable to every SDK, for a mistake only the caller could fix — and the same typo already answered 422 on the build routes, which catch those classes explicitly, so one input produced two verdicts depending on the route. Two dedicated classes now carry `ErrorDomain.INPUT` and a `CHANGE_INPUT` action: `EntryPipeNotFoundError` when the entry lookup resolves nothing, `EntryPipeAmbiguousError` when a bare code is declared in several domains. Both subclass the errors they replace, so every existing catcher still matches, and every presentation derives from the domain at once — 422 on the API, the classification the agent CLI reports, the hosted status. Both are caller-facing copy that survives STRICT disclosure, since the message names only the caller's own `pipe_code` and the qualified `pipe_ref`s of bundles they loaded themselves — which is the only thing that makes an ambiguity actionable. In-body resolution is deliberately untouched: `get_optional_pipe` / `get_required_pipe` keep raising the undomained classes, because a ref written inside a bundle is not a caller's input mistake. The one crossing case is translated rather than left to leak: an ambiguous `alias->bare_code` is detected one frame down, inside the alias-scoped search, and the entry lookup re-raises it as `EntryPipeAmbiguousError` keeping the original as its cause. The agent CLI's `validate` arms stopped hardcoding `PipeNotFoundError` as the reported `error_type` and now name the class that actually failed, which is where the domain and hint on that envelope come from.
- **A typo'd `--pipe` / `pipe_ref` slice selector is reported as the caller's mistake too**: bundle validation narrowed to one pipe (`pipelex-agent validate bundle --pipe`, its `validate method --pipe` twin, and the `pipe_ref` field the build routes forward) raised the undomained `PipeNotFoundError` when the selector named nothing in the bundle, so the same caller typo that answers 422 through the entry lookups came back as a 500 through this one — the divergence the entry-pipe classification above was meant to remove, left in place one module over. The selector is entry-shaped input by the same definition (a code a human typed at an entry point), so the miss now raises `EntryPipeNotFoundError` with its `INPUT` domain and `CHANGE_INPUT` action, keeping the raise site's own bundle-scoped wording, which names only the codes the caller supplied. It is a subclass of the class it replaces, so every existing catcher still matches; the hand-written 422 mapping the public runner API keeps for this case becomes redundant once a release carries the change.
- **A `native.JSON` input produced a fill-in template the runtime refused**: the input-form deriver grouped `native.JSON` with the three natives the standard calls structureless — `Dynamic`, `Anything` and `Composite` — so its descriptor node was `unknown`, which every inputs-template projection renders as the empty object. But `JSON` is not structureless: its pinned blueprint carries a required `json_obj`, and `JSONContent` refuses a payload without it, so the template a caller was handed for a `JSON` input could never be filled in and submitted. It now expands through the same pinned-structure arm as `Date`, `Html`, `Page`, `TextAndImages` and `SearchResult`, rendering `{"json_obj": {}}`, which the input shaper takes back. The engine's own `_pinned_structure` already drew the line in exactly this place — it returns a structure for `JSON` and `None` for the other three — so the deriver is now consistent with it, and the empty object stays the honest rendering for the natives that genuinely withhold their shape. The shared projection corpus gained a pipe covering all three of the natives it can reach at a slot position, which is where the defect had been sitting unmeasured. See [Projection Fixture Corpus](contribute/generate-projection-corpus.md).
+- **A `native.Dynamic` input referenced in a `PipeLLM` prompt no longer fails dry-run validation demanding `ImageContent`**: the prompt analyzers decided whether a variable is a direct image (or document) by asking `is_compatible`, and `Dynamic`'s universal compatibility short-circuits that to `True` — so the analyzer recorded a direct image reference for a slot whose content the caller decides, and every legal bundle mentioning a `Dynamic` input in a prompt was refused at validation. A dynamic root concept now gets no static classification in either analyzer — the question is "is this definitely an X", which universal compatibility cannot answer — matching the guard shape the codebase already used in `PipeLLM.validate_output_with_library` and the input shaper. A `Dynamic` prompt variable renders as text, as it already did in `PipeCompose` templates; authors who want image or document attachment declare `Image`/`Document` (or a refining concept), and `| with_images` on a `Dynamic` variable stays refused. The document analyzer carried the identical latent bug, previously masked by the image misclassification, and gets the same guard. The same analyzer serves `PipeImgGen`'s positive and negative prompts, so the rule lands there too. One live behaviour is given up deliberately and is worth knowing: a `Dynamic` slot does not force `DynamicContent` — the input shaper hands the raw value to the bottom-up factory, so a caller passing an `ImageContent` into a `Dynamic` slot produced a `native.Image` stuff, which the direct reference then attached to the prompt. Attachment is now decided from the declared concept alone, so that image renders as text instead; declare the slot `Image` to keep it reaching the model. See [PipeLLM](building-methods/pipes/pipe-operators/PipeLLM.md).
- **A bad pipeline-input url survives STRICT disclosure as caller-facing copy**: `PipelineInputContentError` declared `caller_facing_message = True`, which is the name of the *report field*, not of the class-level flag that sets it (`_authors_caller_facing_message`). The declaration therefore did nothing, and the message a caller who simply omitted a url needs to see — the one listing the accepted schemes — was replaced by the generic internal-error placeholder on any surface serializing in STRICT mode, exactly the redaction the declaration was written to prevent. Repairing the flag also revealed why the class could not carry it as it stood: its other raise site reports an unreadable local path, and that message names the resolved path and the `OSError` subclass that rejected it, which would have made STRICT answer "does this server-side path exist, and may the runner read it?" for any authenticated caller of a deployment leaving `is_upload_local_content_enabled` on. The family is therefore split: `PipelineInputUrlMissingError` carries the blank-url message and is caller-facing, while `PipelineInputContentError` keeps the unreadable-path message and stays redacted. Both are pinned in the class-level-metadata sweep, in both directions.
- **The projection corpus's divergence gate sees every engine/projection difference**: `pipelex-dev generate-projection-corpus` states that every difference between the reference projection and the engine's own renderer lands in a declared class, and that an undeclared one fails the command — but the walk missed differences in one direction and absorbed regressions into existing classes in the other, so a projection defect could pass generation green and be committed into `mthds-js` and `mthds-python` as contract bytes. The dict walk iterated only the projected keys, so a field the projection stopped rendering was never compared at all; it is now recorded under an id deliberately absent from the declared reasons, which makes the record refuse rather than pass. The file-leaf arm returned on the expansion without ever comparing the URL both sides carry, where plain recursion would have handed a regressed placeholder to the url-named-text class that absorbs it; the shared keys are now compared strictly. And `fixed-count-honoured` fired on any list longer than the engine's single element, which cannot tell an honoured `Concept[N]` from a wrong count or from a variable `[]` slot wrongly duplicated; it now consults the slot's declared `item_count`, taken from the descriptor. Separately, a pipe declaring no inputs used to abort the whole capture — an empty input form is valid and the projection renders it as `{}`, but the engine's renderer refuses one — and is now captured from the projection alone, with no engine half and nothing compared. Re-measuring the same corpus produces bytes identical to the committed capture, with the same classes at the same counts. See [Projection Fixture Corpus](contribute/generate-projection-corpus.md).
- **`--help` works again on every `method` subcommand**: `pipelex validate method`, `pipelex run method`, the three `pipelex build ... method` commands and their agent-CLI twins all crashed with a rich `MarkupError` traceback instead of printing their help. The method-address example in each command's `NAME` argument help — `github.com/owner/repo[/name][@tag]` — reaches rich's markup parser through typer, which read `[/name]` as a closing tag with no opening one. The brackets are escaped at each of the sites, so the example renders as written and the help exits zero.
diff --git a/docs/building-methods/pipes/pipe-operators/PipeLLM.md b/docs/building-methods/pipes/pipe-operators/PipeLLM.md
index 26d4e99b4..35b26007b 100644
--- a/docs/building-methods/pipes/pipe-operators/PipeLLM.md
+++ b/docs/building-methods/pipes/pipe-operators/PipeLLM.md
@@ -209,6 +209,14 @@ Analyze the document and explain how it relates to the context: $reference_doc
"""
```
+## Dynamic Inputs Render as Text
+
+A `native.Dynamic` input referenced in a prompt renders as text through the template, exactly as it does in a `PipeCompose` template. `Dynamic` is compatible with every concept by design, but compatibility is not identity: the prompt analyzers only attach a variable as an image or document when it is *definitely* one, and a dynamic concept's shape is unknowable statically. If you want image or document attachment, declare `Image` / `Document` (or a concept refining them) instead. Consequently, `| with_images` on a `Dynamic` variable is refused — statically unknowable nested images are refused, not guessed.
+
+This holds even when the value you supply at run time *is* an image. A `Dynamic` slot does not force `DynamicContent`: the input shaper hands the raw value to the bottom-up factory, so passing an `ImageContent` (or an explicit `{"concept": "native.Image", ...}` form) into a `Dynamic` slot produces a `native.Image` stuff. That image is still **not** attached to the prompt, because attachment is decided from the declared concept, not from what the caller happened to pass. Declare the slot `Image` when you want the image to reach the model.
+
+The same rule governs `PipeImgGen` prompts (positive and negative), which use the same prompt analyzer.
+
## Configuration
`PipeLLM` is configured in your pipeline's `.mthds` file.
diff --git a/pipelex/pipe_operators/llm/template_document_analyzer.py b/pipelex/pipe_operators/llm/template_document_analyzer.py
index 9a9df34f6..b4e99a682 100644
--- a/pipelex/pipe_operators/llm/template_document_analyzer.py
+++ b/pipelex/pipe_operators/llm/template_document_analyzer.py
@@ -143,6 +143,12 @@ class TemplateDocumentAnalyzer:
Tuple of (is_document_content, is_list_of_documents)
or None if type cannot be resolved
"""
+ # A dynamic concept gets no static classification — same rule as the image analyzer:
+ # `is_compatible` short-circuits to True on Dynamic, and reading that as "definitely a
+ # document" would demand DocumentContent from a slot the runtime fills dynamically.
+ if NativeConceptCode.is_dynamic_concept(concept_code=root_concept.code):
+ return None
+
native_document_concept = get_native_concept(NativeConceptCode.DOCUMENT)
# For simple variable references (no dots after root)
diff --git a/pipelex/pipe_operators/shared/template_image_analyzer.py b/pipelex/pipe_operators/shared/template_image_analyzer.py
index efc0d6ffb..2b10d66fc 100644
--- a/pipelex/pipe_operators/shared/template_image_analyzer.py
+++ b/pipelex/pipe_operators/shared/template_image_analyzer.py
@@ -211,6 +211,14 @@ class TemplateImageAnalyzer:
Tuple of (is_image_content, is_list_of_images, has_nested_images, nested_paths)
or None if type cannot be resolved
"""
+ # A dynamic concept gets no static classification: the question here is "is this
+ # definitely an image", and Dynamic's universal compatibility answers a different
+ # question — `is_compatible` short-circuits to True on it before any structural tier.
+ # It renders as text through the template; an author who wants image attachment
+ # declares Image (or a refining concept).
+ if NativeConceptCode.is_dynamic_concept(concept_code=root_concept.code):
+ return (False, False, False, None)
+
native_image_concept = get_native_concept(NativeConceptCode.IMAGE)
# For simple variable references (no dots after root)
diff --git a/tests/integration/pipelex/pipeline/test_bundle_validator.py b/tests/integration/pipelex/pipeline/test_bundle_validator.py
index 8d6d00cb1..b5f0987d7 100644
--- a/tests/integration/pipelex/pipeline/test_bundle_validator.py
+++ b/tests/integration/pipelex/pipeline/test_bundle_validator.py
@@ -136,6 +136,25 @@ input_item_name = "item"
"""
+# A PipeLLM referencing a native.Dynamic input in its prompt — the shape that used to fail
+# dry-run validation: the image analyzer read Dynamic's universal compatibility as identity,
+# recorded a DIRECT image reference, and the prompt assembler then demanded ImageContent from a
+# slot the dry run fills with DynamicContent. Both prompt markers are covered.
+_BV_DYNAMIC_DOMAIN = "bundle_validator_dynamic"
+_BV_DYNAMIC_MTHDS = f"""
+domain = "{_BV_DYNAMIC_DOMAIN}"
+description = "Bundle with a Dynamic input referenced in a PipeLLM prompt"
+main_pipe = "process_dynamic"
+
+[pipe.process_dynamic]
+type = "PipeLLM"
+description = "Reference a Dynamic input with both prompt markers"
+inputs = {{ dynamic_in = "Dynamic" }}
+output = "Text"
+prompt = "Process this:\\n@dynamic_in\\nand inline $dynamic_in"
+"""
+
+
@pytest.mark.asyncio(loop_scope="class")
class TestBundleValidatorIntegration:
async def test_validate_pipes_success_real_path(self) -> None:
@@ -152,6 +171,23 @@ class TestBundleValidatorIntegration:
library_manager.teardown(library_id=library_id)
clear_current_library()
+ async def test_dynamic_input_in_prompt_dry_runs_clean(self) -> None:
+ """A PipeLLM referencing a Dynamic input in its prompt passes dry-run validation:
+ a dynamic concept gets no static image (or document) classification, so the variable
+ renders as text through the template.
+ """
+ library_manager = get_library_manager()
+ library_id = "bv_dynamic_lib"
+ _, qualified_main_pipe = acquire_library(library_id=library_id, mthds_contents=[_BV_DYNAMIC_MTHDS])
+ try:
+ assert qualified_main_pipe is not None
+ pipe = get_required_entry_pipe(pipe_code=qualified_main_pipe)
+ results = await BundleValidator().validate_pipes([pipe], library_id=library_id)
+ assert results[pipe.pipe_ref].status.is_success
+ finally:
+ library_manager.teardown(library_id=library_id)
+ clear_current_library()
+
async def test_consecutive_sweeps_use_distinct_run_ids(self, mocker: MockerFixture) -> None:
# Each sweep dry-runs its pipes under a UNIQUE per-sweep pipeline run id (a `dry_run_`-prefixed
# uuid) threaded into prepare_pipe_job. With the live registry removed (usage rides on PipeOutput),
diff --git a/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_document_analyzer.py b/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_document_analyzer.py
index bfd73d87a..4077706fc 100644
--- a/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_document_analyzer.py
+++ b/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_document_analyzer.py
@@ -28,6 +28,20 @@ class TestTemplateDocumentAnalyzer:
assert result[0].variable_path == "document"
assert result[0].kind == DocumentReferenceKind.DIRECT
+ def test_dynamic_input_yields_no_document_reference(self, load_test_library: Callable[[list[Path]], None]) -> None:
+ """A Dynamic input referenced plainly is never statically a document — same rule as the
+ image analyzer: universal compatibility is not identity.
+ """
+ load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
+
+ result = TemplateDocumentAnalyzer.analyze_template_for_documents(
+ template_source="Process this:\n@dynamic_in\nand inline $dynamic_in",
+ input_specs={"dynamic_in": "Dynamic"},
+ domain_code="test_pipes",
+ )
+
+ assert result == []
+
def test_document_list_input_creates_direct_list_reference(self, load_test_library: Callable[[list[Path]], None]) -> None:
"""Test that Document[] input creates a DIRECT_LIST reference."""
load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
diff --git a/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_image_analyzer.py b/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_image_analyzer.py
index c631699b1..2e8653df9 100644
--- a/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_image_analyzer.py
+++ b/tests/unit/pipelex/pipe_operators/pipe_llm/test_template_image_analyzer.py
@@ -240,6 +240,37 @@ class TestTemplateImageAnalyzer:
kinds = {ref.kind for ref in result}
assert kinds == {ImageReferenceKind.DIRECT, ImageReferenceKind.NESTED}
+ # --------------------------------------------------------------------------
+ # Dynamic concepts: no static classification
+ # --------------------------------------------------------------------------
+
+ def test_dynamic_input_yields_no_image_reference(self, load_test_library: Callable[[list[Path]], None]) -> None:
+ """A Dynamic input referenced plainly is never statically an image: its universal
+ compatibility answers a different question than "is this definitely an image".
+ """
+ load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
+
+ result = TemplateImageAnalyzer.analyze_template_for_images(
+ template_source="Process this:\n@dynamic_in\nand inline $dynamic_in",
+ input_specs={"dynamic_in": "Dynamic"},
+ domain_code="test_pipes",
+ )
+
+ assert result == []
+
+ def test_with_images_on_dynamic_raises(self, load_test_library: Callable[[list[Path]], None]) -> None:
+ """`| with_images` on a Dynamic variable is refused: statically unknowable nested images
+ are refused, not guessed.
+ """
+ load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
+
+ with pytest.raises(WithImagesFilterError):
+ TemplateImageAnalyzer.analyze_template_for_images(
+ template_source="Process: {{ dynamic_in | with_images }}",
+ input_specs={"dynamic_in": "Dynamic"},
+ domain_code="test_pipes",
+ )
+
class TestValidateUnusedInputs:
"""Tests for TemplateImageAnalyzer.validate_unused_inputs()."""