Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix/dynamic-concept-ref (#844)

Thomas Hebrard085a432d2123
diff --git a/.github/workflows/publish-pypi.yml b/.github/workflows/publish-pypi.yml
index bdfaa035a..061c84236 100644
--- a/.github/workflows/publish-pypi.yml
+++ b/.github/workflows/publish-pypi.yml
@@ -5,7 +5,7 @@ on:
types: [closed]
branches:
- main
- - 'pre-release/v*'
+ - "pre-release/v*"
jobs:
build:
@@ -93,19 +93,19 @@ jobs:
run: |
VERSION="${{ env.VERSION }}"
echo "Extracting changelog for version v$VERSION"
-
+
# Find the start of the current version section
START_LINE=$(grep -n "## \[v$VERSION\] - " CHANGELOG.md | cut -d: -f1)
-
+
if [ -z "$START_LINE" ]; then
echo "Warning: No changelog entry found for version v$VERSION"
echo "CHANGELOG_NOTES=" >> $GITHUB_ENV
exit 0
fi
-
+
# Find the start of the next version section (previous version)
NEXT_VERSION_LINE=$(tail -n +$((START_LINE + 1)) CHANGELOG.md | grep -n "^## \[v.*\] - " | head -1 | cut -d: -f1)
-
+
if [ -z "$NEXT_VERSION_LINE" ]; then
# No next version found, extract from current version till end of file
CHANGELOG_CONTENT=$(tail -n +$START_LINE CHANGELOG.md)
@@ -114,15 +114,15 @@ jobs:
END_LINE=$((START_LINE + NEXT_VERSION_LINE - 1))
CHANGELOG_CONTENT=$(sed -n "$START_LINE,$((END_LINE - 1))p" CHANGELOG.md)
fi
-
+
# Clean up the content but preserve the blank line after the header
# First, get the header line and add a blank line after it
HEADER_LINE=$(echo "$CHANGELOG_CONTENT" | head -1)
CONTENT_LINES=$(echo "$CHANGELOG_CONTENT" | tail -n +2 | sed '/^$/d' | sed 's/^[[:space:]]*//' | sed 's/[[:space:]]*$//')
-
+
# Combine header + blank line + content
CHANGELOG_CONTENT=$(printf "%s\n\n%s" "$HEADER_LINE" "$CONTENT_LINES")
-
+
# Escape for GitHub Actions
echo "CHANGELOG_NOTES<<EOF" >> $GITHUB_ENV
echo "$CHANGELOG_CONTENT" >> $GITHUB_ENV
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4cd1bf1b5..b29d8fa8e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,19 @@
# Changelog
+## [Unreleased]
+
+### Added
+
+- **`--dynamic-output-concept` / `-O` flag on `pipelex run`.** All three subcommands (`run bundle`, `run pipe`, `run method`) accept a concept ref (e.g. `document_qa.ReferenceCount`) used to resolve a pipe whose output is declared as `Dynamic`. Threaded through `_run_core.execute_run` to `PipelexRunner.execute_pipeline(dynamic_output_concept_ref=...)`. Until now, Dynamic-output pipes were only callable from the Python runner.
+
+- **Line-length-safe wrapping in the structures generator.** `pipelex build structures` now wraps long descriptions so every emitted line stays under the 150-char ruff limit. Long class docstrings become a multi-line triple-quoted block; long Field descriptions become a parenthesized implicit-string-concatenation block (`description=("first chunk " "second chunk")`). Short descriptions still emit the compact single-line form. New unit tests in `tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.py` cover both lengths and the combined long-everything case. Previously, descriptions above ~140 chars produced files that failed `ruff check` with E501.
+
+### Fixed
+
+- **`PipeLLM` Dynamic-output detection compared the wrong fields.** `pipe_llm.py` checked `self.output.concept.code == "native.Dynamic"`, but `concept.code` is the bare code (`"Dynamic"`) not the qualified ref. The check never matched, so when a caller passed `dynamic_output_concept_ref` for a `Dynamic`-output pipe, the resolver branch was skipped silently: the output structure stayed `DynamicContent` (an empty `StuffContent` subclass), the LLM produced JSON shaped like the requested concept, and the result deserialized to `{}`. Detection now uses `concept.code == NativeConceptCode.DYNAMIC and concept.domain_code == SpecialDomain.NATIVE`.
+
+- **Dynamic-output concept resolution rejected qualified refs.** When the runtime override was supplied (e.g. `"document_qa.ReferenceCount"`), the previous code called `make_concept_ref_with_domain(domain_code=self.domain_code, concept_code=output_concept_ref)`, producing `"document_qa.document_qa.ReferenceCount"` and a missing-concept lookup. Now uses `make_concept_ref_with_domain_from_concept_ref_or_code`, which extracts the domain from the input when it's already qualified and falls back to the pipe's domain only for bare codes. Callers can pass either form.
+
## [v0.25.0] - 2026-04-28
### Added
diff --git a/pipelex/cli/commands/run/_run_core.py b/pipelex/cli/commands/run/_run_core.py
index 95c69232f..a0bed6c81 100644
--- a/pipelex/cli/commands/run/_run_core.py
+++ b/pipelex/cli/commands/run/_run_core.py
@@ -51,6 +51,7 @@ async def _execute_run(
dry_run: bool,
mock_inputs: bool,
library_dir: list[str] | None,
+ dynamic_output_concept_ref: str | None = None,
) -> None:
"""Core async execution logic for running a pipe.
@@ -127,6 +128,7 @@ async def _execute_run(
pipe_code=pipe_code,
mthds_contents=[mthds_content] if mthds_content else None,
inputs=pipeline_inputs,
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
)
pipe_output = response.pipe_output
except PipelineExecutionError as exc:
@@ -244,6 +246,7 @@ def execute_run(
mock_inputs: bool,
library_dir: list[str] | None,
telemetry_command_label: str = COMMAND,
+ dynamic_output_concept_ref: str | None = None,
) -> None:
"""Synchronous entry point that wraps the async execution with Pipelex setup/teardown.
@@ -271,6 +274,7 @@ def execute_run(
dry_run=dry_run,
mock_inputs=mock_inputs,
library_dir=library_dir,
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
)
)
diff --git a/pipelex/cli/commands/run/bundle_cmd.py b/pipelex/cli/commands/run/bundle_cmd.py
index 33971145a..2b7155184 100644
--- a/pipelex/cli/commands/run/bundle_cmd.py
+++ b/pipelex/cli/commands/run/bundle_cmd.py
@@ -69,6 +69,14 @@ def run_bundle_cmd(
list[str] | None,
typer.Option("--library-dir", "-L", help="Directory to search for pipe definitions (.mthds files). Can be specified multiple times."),
] = None,
+ dynamic_output_concept_ref: Annotated[
+ str | None,
+ typer.Option(
+ "--dynamic-output-concept",
+ "-O",
+ help="Concept ref (e.g. 'document_qa.ReferenceCount') used to resolve a pipe whose output is declared as 'Dynamic'.",
+ ),
+ ] = None,
) -> None:
"""Run a pipeline from a bundle file (.mthds) or pipeline directory.
@@ -160,4 +168,5 @@ def run_bundle_cmd(
mock_inputs=mock_inputs,
library_dir=library_dir,
telemetry_command_label=f"{COMMAND} bundle",
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
)
diff --git a/pipelex/cli/commands/run/method_cmd.py b/pipelex/cli/commands/run/method_cmd.py
index 9fd17a46b..1954eb19b 100644
--- a/pipelex/cli/commands/run/method_cmd.py
+++ b/pipelex/cli/commands/run/method_cmd.py
@@ -68,6 +68,14 @@ def run_method_cmd(
list[str] | None,
typer.Option("--library-dir", "-L", help="Directory to search for pipe definitions (.mthds files). Can be specified multiple times."),
] = None,
+ dynamic_output_concept_ref: Annotated[
+ str | None,
+ typer.Option(
+ "--dynamic-output-concept",
+ "-O",
+ help="Concept ref (e.g. 'document_qa.ReferenceCount') used to resolve a pipe whose output is declared as 'Dynamic'.",
+ ),
+ ] = None,
) -> None:
"""Run an installed method by name.
@@ -130,4 +138,5 @@ def run_method_cmd(
mock_inputs=mock_inputs,
library_dir=effective_library_dir,
telemetry_command_label=f"{COMMAND} method",
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
)
diff --git a/pipelex/cli/commands/run/pipe_cmd.py b/pipelex/cli/commands/run/pipe_cmd.py
index 67c7926fb..90ba5cfc1 100644
--- a/pipelex/cli/commands/run/pipe_cmd.py
+++ b/pipelex/cli/commands/run/pipe_cmd.py
@@ -65,6 +65,14 @@ def run_pipe_cmd(
list[str] | None,
typer.Option("--library-dir", "-L", help="Directory to search for pipe definitions (.mthds files). Can be specified multiple times."),
] = None,
+ dynamic_output_concept_ref: Annotated[
+ str | None,
+ typer.Option(
+ "--dynamic-output-concept",
+ "-O",
+ help="Concept ref (e.g. 'document_qa.ReferenceCount') used to resolve a pipe whose output is declared as 'Dynamic'.",
+ ),
+ ] = None,
) -> None:
"""Run a pipe by code.
@@ -125,4 +133,5 @@ def run_pipe_cmd(
mock_inputs=mock_inputs,
library_dir=library_dir,
telemetry_command_label=f"{COMMAND} pipe",
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
)
diff --git a/pipelex/core/concepts/concept_factory.py b/pipelex/core/concepts/concept_factory.py
index 648210f78..eb188c2b3 100644
--- a/pipelex/core/concepts/concept_factory.py
+++ b/pipelex/core/concepts/concept_factory.py
@@ -214,9 +214,9 @@ class ConceptFactory:
return f"{domain_code}.{concept_code}"
@classmethod
- def make_concept_ref_with_domain_from_concept_ref_or_code(cls, domain_code: str, concept_sring_or_code: str) -> str:
+ def make_concept_ref_with_domain_from_concept_ref_or_code(cls, domain_code: str, concept_ref_or_code: str) -> str:
input_domain_and_code = cls.make_domain_and_concept_code_from_concept_ref_or_code(
- concept_ref_or_code=concept_sring_or_code,
+ concept_ref_or_code=concept_ref_or_code,
domain_code=domain_code,
)
diff --git a/pipelex/core/concepts/structure_generation/generator.py b/pipelex/core/concepts/structure_generation/generator.py
index cbedb669b..635e83042 100644
--- a/pipelex/core/concepts/structure_generation/generator.py
+++ b/pipelex/core/concepts/structure_generation/generator.py
@@ -1,4 +1,5 @@
import ast
+import textwrap
from datetime import datetime
from enum import Enum
from typing import Any, List, Literal, Optional
@@ -13,6 +14,11 @@ from pipelex.core.concepts.structure_generation.exceptions import ConceptStructu
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.core.stuffs.stuff_content import StuffContent
+# Target line length for emitted code (matches the project's ruff config).
+# Each emitted line stays under this so ruff E501 doesn't trigger on long descriptions.
+_MAX_LINE_LENGTH = 150
+_WRAP_WIDTH = 120 # conservative: leaves headroom for indentation and syntax overhead
+
class ConceptClassInfo:
"""Information about a concept's structure class for import generation."""
@@ -184,6 +190,42 @@ class StructureGenerator:
return self._escape_string_for_python(value)
return repr(value)
+ def _format_class_docstring(self, docstring: str, indent: str = " ") -> str:
+ """Render a class docstring, wrapping it across multiple lines when the single-line
+ form would exceed _MAX_LINE_LENGTH.
+ """
+ single_line = f'{indent}"""{docstring}"""'
+ if len(single_line) <= _MAX_LINE_LENGTH:
+ return single_line
+ # Wrap the docstring text. The first wrapped line follows """, subsequent lines are
+ # plain indented text, and the closing """ sits on its own line.
+ wrap_width = _WRAP_WIDTH
+ wrapped_lines = textwrap.wrap(docstring, width=wrap_width, break_long_words=False, break_on_hyphens=False)
+ if not wrapped_lines:
+ return single_line
+ body = "\n".join(f"{indent}{line}" for line in wrapped_lines)
+ return f'{indent}"""\n{body}\n{indent}"""'
+
+ def _format_field_description(self, description: str) -> str:
+ """Render the `description=` argument for a Field call. For short values, emit a
+ single-line `description="..."`. For long values, emit a parenthesized
+ implicit-string-concatenation block so each emitted line stays under the line limit.
+ """
+ escaped = self._escape_string_for_python(description)
+ if len(escaped) <= _WRAP_WIDTH:
+ return f"description={escaped}"
+ chunks = textwrap.wrap(description, width=_WRAP_WIDTH, break_long_words=False, break_on_hyphens=False)
+ if len(chunks) <= 1:
+ return f"description={escaped}"
+ # Join trailing space on each chunk except the last so the concatenation reads
+ # naturally; textwrap dropped the original spaces.
+ chunk_literals: list[str] = []
+ for index, chunk in enumerate(chunks):
+ content = chunk if index == len(chunks) - 1 else f"{chunk} "
+ chunk_literals.append(self._escape_string_for_python(content))
+ body = "\n".join(chunk_literals)
+ return f"description=(\n{body}\n)"
+
def _generate_class_source_code_from_blueprint(
self,
class_name: str,
@@ -222,7 +264,7 @@ class StructureGenerator:
# Generate class header with docstring (use class name if no description provided)
docstring = description or f"Generated {class_name} class"
- class_header = f'class {class_name}({base_class}):\n """{docstring}"""\n'
+ class_header = f"class {class_name}({base_class}):\n{self._format_class_docstring(docstring)}\n"
# Generate fields
field_definitions: list[str] = []
@@ -261,7 +303,7 @@ class StructureGenerator:
python_type = f"Optional[{python_type}]"
# Generate Field parameters
- field_params = [f"description={self._escape_string_for_python(field_blueprint.description)}"]
+ field_params = [self._format_field_description(field_blueprint.description)]
if field_blueprint.required:
if field_blueprint.default_value is not None:
@@ -429,7 +471,7 @@ class StructureGenerator:
python_type = f"Optional[{python_type}]"
# Generate Field parameters
- field_params = [f"description={self._escape_string_for_python(description)}"]
+ field_params = [self._format_field_description(description)]
if required:
if default_value is not None:
diff --git a/pipelex/core/pipes/inputs/input_stuff_specs_factory.py b/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
index 93e55520c..a2b0944c5 100644
--- a/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
+++ b/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
@@ -83,7 +83,7 @@ class InputStuffSpecsFactory:
concept_ref_with_domain = ConceptFactory.make_concept_ref_with_domain_from_concept_ref_or_code(
domain_code=domain_code,
- concept_sring_or_code=concept_ref_or_code,
+ concept_ref_or_code=concept_ref_or_code,
)
# Determine multiplicity
diff --git a/pipelex/pipe_operators/llm/pipe_llm.py b/pipelex/pipe_operators/llm/pipe_llm.py
index e97df2024..4a6077a96 100644
--- a/pipelex/pipe_operators/llm/pipe_llm.py
+++ b/pipelex/pipe_operators/llm/pipe_llm.py
@@ -23,6 +23,7 @@ from pipelex.core.pipes.exceptions import PipeValidationError, PipeValidationErr
from pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecs
from pipelex.core.pipes.inputs.input_stuff_specs_factory import InputStuffSpecsFactory
from pipelex.core.pipes.pipe_output import PipeOutput
+from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec
from pipelex.core.pipes.validation import is_input_used_by_variables, is_variable_satisfied_by_inputs
from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity
from pipelex.core.stuffs.list_content import ListContent
@@ -152,6 +153,32 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):
def required_variables(self) -> set[str]:
return {variable_name for variable_name in self.llm_prompt_spec.required_variables() if not variable_name.startswith("_")}
+ def resolve_dynamic_output_stuff_spec(self, pipe_run_params: PipeRunParams) -> StuffSpec:
+ """Return the `StuffSpec` to use for this run. When the pipe's declared output is
+ `native.Dynamic`, the actual concept is resolved from the run params (override
+ on `pipe_run_params.dynamic_output_concept_ref`, with a legacy fallback on
+ `pipe_run_params.params[DYNAMIC_OUTPUT_CONCEPT]`, defaulting to `native.Text`)
+ and returned in a copy of `self.output`. Otherwise returns `self.output` unchanged.
+
+ Pure: never mutates `self`. The same pipe instance can therefore be reused across
+ runs with different `dynamic_output_concept_ref` values without the first run's
+ choice sticking.
+ """
+ # TODO: DYNAMIC_OUTPUT_CONCEPT should not be a key in `params`; promote to an
+ # attribute on PipeRunParams and drop the params-key fallback.
+ if self.output.concept.code != NativeConceptCode.DYNAMIC or self.output.concept.domain_code != SpecialDomain.NATIVE:
+ return self.output
+ output_concept_ref = pipe_run_params.dynamic_output_concept_ref or pipe_run_params.params.get(PipeRunParamKey.DYNAMIC_OUTPUT_CONCEPT)
+ if not output_concept_ref:
+ output_concept_ref = SpecialDomain.NATIVE + "." + NativeConceptCode.TEXT
+ resolved_concept = get_required_concept(
+ concept_ref=ConceptFactory.make_concept_ref_with_domain_from_concept_ref_or_code(
+ domain_code=self.domain_code,
+ concept_ref_or_code=output_concept_ref,
+ ),
+ )
+ return self.output.model_copy(update={"concept": resolved_concept})
+
@override
async def _live_run_operator_pipe(
self,
@@ -163,18 +190,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):
) -> PipeLLMOutput:
content_generator = content_generator or get_content_generator()
# interpret / unwrap the arguments
- output_stuff_spec = self.output
- if self.output.concept.code == SpecialDomain.NATIVE + "." + NativeConceptCode.DYNAMIC:
- # TODO: This DYNAMIC_OUTPUT_CONCEPT should not be a field in the params attribute of PipeRunParams.
- # It should be an attribute of PipeRunParams.
- output_concept_code = pipe_run_params.dynamic_output_concept_code or pipe_run_params.params.get(PipeRunParamKey.DYNAMIC_OUTPUT_CONCEPT)
-
- if not output_concept_code:
- output_concept_code = SpecialDomain.NATIVE + "." + NativeConceptCode.TEXT
- else:
- output_stuff_spec.concept = get_required_concept(
- concept_ref=ConceptFactory.make_concept_ref_with_domain(domain_code=self.domain_code, concept_code=output_concept_code),
- )
+ output_stuff_spec = self.resolve_dynamic_output_stuff_spec(pipe_run_params=pipe_run_params)
multiplicity_resolution = output_multiplicity_to_apply(
base_multiplicity=self.output_multiplicity,
@@ -302,7 +318,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):
output_structure_prompt: str | None = None
if get_config().cogt.llm_config.is_structure_prompt_enabled:
output_structure_prompt = await get_output_structure_prompt(
- concept_ref=pipe_run_params.dynamic_output_concept_code or output_stuff_spec.concept.concept_ref,
+ concept_ref=pipe_run_params.dynamic_output_concept_ref or output_stuff_spec.concept.concept_ref,
is_with_preliminary_text=is_with_preliminary_text,
)
llm_prompt_1_for_object = await self.llm_prompt_spec.make_llm_prompt(
diff --git a/pipelex/pipe_operators/pipe_operator.py b/pipelex/pipe_operators/pipe_operator.py
index c73562cd4..eff33318e 100644
--- a/pipelex/pipe_operators/pipe_operator.py
+++ b/pipelex/pipe_operators/pipe_operator.py
@@ -40,7 +40,10 @@ class PipeOperator(PipeAbstract, Generic[PipeOperatorOutputType]):
)
main_stuff = pipe_output.main_stuff
- output_concept_code = self.output.concept.code
+ # Read the concept from main_stuff, not self.output: when the pipe declares
+ # `Dynamic` output, the runtime-resolved concept lives on main_stuff, while
+ # self.output.concept stays Dynamic across runs.
+ output_concept_code = main_stuff.concept.code
output_concept_with_multiplicity = f"[bold green]{output_concept_code}[/bold green]"
if main_stuff.is_list:
list_content: ListContent[StuffContent] = main_stuff.as_list_content() # pyright: ignore[reportUnknownVariableType, reportUnknownMemberType]
diff --git a/pipelex/pipe_run/pipe_run_params.py b/pipelex/pipe_run/pipe_run_params.py
index 1ae739dfc..dafb079ff 100644
--- a/pipelex/pipe_run/pipe_run_params.py
+++ b/pipelex/pipe_run/pipe_run_params.py
@@ -139,7 +139,7 @@ class PipeRunParams(BaseModel):
final_stuff_code: str | None = None
is_with_preliminary_text: bool | None = None
output_multiplicity: VariableMultiplicity | None = None
- dynamic_output_concept_code: str | None = None
+ dynamic_output_concept_ref: str | None = None
batch_params: BatchParams | None = None
params: dict[str, Any] = Field(default_factory=dict)
diff --git a/pipelex/pipe_run/pipe_run_params_factory.py b/pipelex/pipe_run/pipe_run_params_factory.py
index 674c5b0e1..4819de965 100644
--- a/pipelex/pipe_run/pipe_run_params_factory.py
+++ b/pipelex/pipe_run/pipe_run_params_factory.py
@@ -12,7 +12,7 @@ class PipeRunParamsFactory:
pipe_run_mode: PipeRunMode = PipeRunMode.LIVE,
pipe_stack_limit: int | None = None,
output_multiplicity: VariableMultiplicity | None = None,
- dynamic_output_concept_code: str | None = None,
+ dynamic_output_concept_ref: str | None = None,
batch_params: BatchParams | None = None,
params: dict[str, Any] | None = None,
) -> PipeRunParams:
@@ -21,7 +21,7 @@ class PipeRunParamsFactory:
run_mode=pipe_run_mode,
pipe_stack_limit=pipe_stack_limit,
output_multiplicity=output_multiplicity,
- dynamic_output_concept_code=dynamic_output_concept_code,
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
batch_params=batch_params,
params=params or {},
)
diff --git a/pipelex/pipeline/pipeline_run_setup.py b/pipelex/pipeline/pipeline_run_setup.py
index 9acfaecc7..b5cbf665d 100644
--- a/pipelex/pipeline/pipeline_run_setup.py
+++ b/pipelex/pipeline/pipeline_run_setup.py
@@ -53,7 +53,7 @@ async def pipeline_run_setup(
inputs: PipelineInputs | WorkingMemory | None = None,
output_name: str | None = None,
output_multiplicity: VariableMultiplicity | None = None,
- dynamic_output_concept_code: str | None = None,
+ dynamic_output_concept_ref: str | None = None,
pipe_run_mode: PipeRunMode | None = None,
search_domain_codes: list[str] | None = None,
user_id: str | None = None,
@@ -100,7 +100,7 @@ async def pipeline_run_setup(
Name of the output slot to write to.
output_multiplicity:
Output multiplicity specification.
- dynamic_output_concept_code:
+ dynamic_output_concept_ref:
Override the dynamic output concept code.
pipe_run_mode:
Pipe run mode: ``PipeRunMode.LIVE`` or ``PipeRunMode.DRY``. If not specified,
@@ -283,7 +283,7 @@ async def pipeline_run_setup(
pipe_run_params = PipeRunParamsFactory.make_run_params(
output_multiplicity=output_multiplicity,
- dynamic_output_concept_code=dynamic_output_concept_code,
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
pipe_run_mode=pipe_run_mode,
)
diff --git a/pipelex/pipeline/runner.py b/pipelex/pipeline/runner.py
index 09505adc2..ce38e497c 100644
--- a/pipelex/pipeline/runner.py
+++ b/pipelex/pipeline/runner.py
@@ -74,7 +74,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):
inputs: PipelineInputs | WorkingMemoryAbstract[Any] | None = None,
output_name: str | None = None,
output_multiplicity: VariableMultiplicity | None = None,
- dynamic_output_concept_code: str | None = None,
+ dynamic_output_concept_ref: str | None = None,
) -> PipelexPipelineExecuteResponse:
"""Execute a pipeline and wait for its completion.
@@ -102,7 +102,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):
Name of the output slot to write to.
output_multiplicity:
Output multiplicity specification.
- dynamic_output_concept_code:
+ dynamic_output_concept_ref:
Override the dynamic output concept code.
Returns:
@@ -139,7 +139,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):
inputs=pipelex_inputs,
output_name=output_name,
output_multiplicity=output_multiplicity,
- dynamic_output_concept_code=dynamic_output_concept_code,
+ dynamic_output_concept_ref=dynamic_output_concept_ref,
pipe_run_mode=self.pipe_run_mode,
search_domain_codes=self.search_domain_codes,
user_id=self.user_id,
@@ -227,6 +227,6 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):
inputs: PipelineInputs | WorkingMemoryAbstract[Any] | None = None,
output_name: str | None = None,
output_multiplicity: VariableMultiplicity | None = None,
- dynamic_output_concept_code: str | None = None,
+ dynamic_output_concept_ref: str | None = None,
) -> PipelineStartResponse[PipeOutput]:
raise NotImplementedError
diff --git a/pyproject.toml b/pyproject.toml
index 0a1962a33..d7638edd6 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -28,7 +28,7 @@ dependencies = [
"json2html>=1.3.0",
"kajson==0.3.1",
"markdown>=3.6",
- "mthds>=0.2.0",
+ "mthds>=0.3.0",
"networkx>=3.4.2",
"openai>=1.108.1",
"opentelemetry-api",
diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
index d4fdbdfd1..c1dbd3ba8 100644
--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
@@ -1,4 +1,3 @@
-# ruff: noqa: E501
"""Unit tests for string escaping in StructureGenerator.
This module tests that field descriptions and default values containing special
@@ -516,7 +515,11 @@ from typing import Optional, List, Dict, Any, Literal
class LongDesc(StructuredContent):
"""Generated LongDesc class"""
- long_field: str = Field(..., description="This is a very long description that contains many words and also has some \\"quoted sections\\" that need to be properly escaped. It goes on and on with more details about the field, including examples like \\"example1\\", \\"example2\\", and \\"example3\\". The description continues with even more information to test the robustness of the escaping mechanism.")
+ long_field: str = Field(..., description=(
+"This is a very long description that contains many words and also has some \\"quoted sections\\" that need to be properly "
+"escaped. It goes on and on with more details about the field, including examples like \\"example1\\", \\"example2\\", and "
+"\\"example3\\". The description continues with even more information to test the robustness of the escaping mechanism."
+))
'''
assert generated_code == expected_code
diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.py
new file mode 100644
index 000000000..4197426ba
--- /dev/null
+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_wrapping.py
@@ -0,0 +1,163 @@
+"""Unit tests for line-length wrapping in StructureGenerator.
+
+Generated structure files must stay under the project's ruff line-length limit
+(150 chars). The generator handles this in two places:
+
+- **Class docstrings**: emitted as a multi-line triple-quoted block when the
+ single-line form would exceed the limit.
+- **Field `description=` arguments**: emitted as a parenthesized
+ implicit-string-concatenation block when the inline `description="..."` would
+ exceed the limit.
+
+For short descriptions, both fall back to the compact single-line form.
+"""
+
+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType
+from pipelex.core.concepts.structure_generation.generator import StructureGenerator
+
+_LINE_LIMIT = 150
+
+
+def _max_line_length(code: str) -> int:
+ return max((len(line) for line in code.splitlines()), default=0)
+
+
+class TestStructureGeneratorWrapping:
+ def test_short_class_docstring_stays_single_line(self):
+ """A short class description fits on one line and is emitted compactly."""
+ blueprint = {
+ "name": ConceptStructureBlueprint(
+ description="Name",
+ type=ConceptStructureBlueprintFieldType.TEXT,
+ required=True,
+ ),
+ }
+ code, _ = StructureGenerator().generate_from_structure_blueprint(
+ class_name="Person",
+ structure_blueprint=blueprint,
+ description="A person.",
+ )
+ assert '"""A person."""' in code
+ assert _max_line_length(code) <= _LINE_LIMIT
+
+ def test_long_class_docstring_wraps_to_multiline(self):
+ """A long class description is split across multiple lines, each under the limit."""
+ long_description = (
+ "Specific output structure for this example's question — how many references in the report come from the "
+ "report's own research center. Demonstrates how to output a typed structure tailored to the question rather "
+ "than a free-form text answer."
+ )
+ blueprint = {
+ "count": ConceptStructureBlueprint(
+ description="Count",
+ type=ConceptStructureBlueprintFieldType.INTEGER,
+ required=True,
+ ),
+ }
+ code, _ = StructureGenerator().generate_from_structure_blueprint(
+ class_name="ReferenceCount",
+ structure_blueprint=blueprint,
+ description=long_description,
+ )
+
+ # Multi-line docstring form: opening """, body lines, closing """ on its own line.
+ assert ' """\n' in code
+ assert '\n """' in code
+ # Every emitted line stays under the limit.
+ assert _max_line_length(code) <= _LINE_LIMIT
+ # The full description content is preserved across the wrapped lines.
+ # Fragments must fit within a single wrapped line (textwrap inserts \n at word boundaries).
+ for fragment in ["Specific output structure", "research center", "free-form text"]:
+ assert fragment in code
+
+ def test_short_field_description_stays_single_line(self):
+ """A short Field description uses the compact `description="..."` form."""
+ blueprint = {
+ "count": ConceptStructureBlueprint(
+ description="The number of references coming from the report's own research center.",
+ type=ConceptStructureBlueprintFieldType.INTEGER,
+ required=True,
+ ),
+ }
+ code, _ = StructureGenerator().generate_from_structure_blueprint(
+ class_name="ReferenceCount",
+ structure_blueprint=blueprint,
+ )
+ assert 'description="The number of references coming from the report\'s own research center."' in code
+ assert "description=(" not in code
+ assert _max_line_length(code) <= _LINE_LIMIT
+
+ def test_long_field_description_wraps_to_implicit_concat(self):
+ """A long Field description becomes a parenthesized implicit-string-concatenation block."""
+ long_description = (
+ "Whether a factual answer could in principle be derived from documents. False for counterfactual, opinion, "
+ "and out_of_scope questions. This field is the central gate that prevents synthesis from forcing a question "
+ "into an answerable type just to be helpful."
+ )
+ blueprint = {
+ "is_answerable_from_documents": ConceptStructureBlueprint(
+ description=long_description,
+ type=ConceptStructureBlueprintFieldType.BOOLEAN,
+ required=True,
+ ),
+ }
+ code, _ = StructureGenerator().generate_from_structure_blueprint(
+ class_name="QuestionAnalysis",
+ structure_blueprint=blueprint,
+ )
+
+ # Parenthesized implicit-concat form was used.
+ assert "description=(" in code
+ # Every emitted line stays under the limit.
+ assert _max_line_length(code) <= _LINE_LIMIT
+ # Each fragment of the original description appears in the generated code
+ # (each within a single chunk — fragments that span chunk boundaries would
+ # be interrupted by the literal `" "` between adjacent string literals).
+ for fragment in [
+ "Whether a factual answer",
+ "out_of_scope questions",
+ "answerable type just to be helpful",
+ ]:
+ assert fragment in code
+
+ def test_long_descriptions_in_long_class_produce_clean_output(self):
+ """End-to-end: a class with a long docstring AND multiple long field descriptions
+ still stays under the line limit on every emitted line.
+ """
+ long_class_doc = (
+ "Structured understanding of the user's question: its type, whether it is answerable from documents at all, "
+ "its decomposition into sub-questions, key entities to search for, alternative phrasings to boost retrieval "
+ "recall, and any ambiguities that may block a confident answer."
+ )
+ blueprint = {
+ "sub_questions": ConceptStructureBlueprint(
+ description=(
+ "Decomposition of a compound question into independently-answerable sub-questions. For simple "
+ "questions, a single-item list containing the original question."
+ ),
+ type=ConceptStructureBlueprintFieldType.LIST,
+ required=True,
+ ),
+ "reformulations": ConceptStructureBlueprint(
+ description=(
+ "Alternative phrasings and synonyms for the question's key terms. Improves retrieval recall when "
+ "documents use different vocabulary."
+ ),
+ type=ConceptStructureBlueprintFieldType.LIST,
+ required=True,
+ ),
+ "ambiguities": ConceptStructureBlueprint(
+ description=(
+ "Aspects of the question that are ambiguous and may require clarification from the user. Empty "
+ "list if the question is fully specified."
+ ),
+ type=ConceptStructureBlueprintFieldType.LIST,
+ required=False,
+ ),
+ }
+ code, _ = StructureGenerator().generate_from_structure_blueprint(
+ class_name="QuestionAnalysis",
+ structure_blueprint=blueprint,
+ description=long_class_doc,
+ )
+ assert _max_line_length(code) <= _LINE_LIMIT
diff --git a/tests/unit/pipelex/pipe_operators/pipe_llm/test_pipe_llm_dynamic_output.py b/tests/unit/pipelex/pipe_operators/pipe_llm/test_pipe_llm_dynamic_output.py
new file mode 100644
index 000000000..e69a939f5
--- /dev/null
+++ b/tests/unit/pipelex/pipe_operators/pipe_llm/test_pipe_llm_dynamic_output.py
@@ -0,0 +1,155 @@
+"""Unit tests for `PipeLLM.resolve_dynamic_output_stuff_spec`.
+
+When a PipeLLM declares `output = "Dynamic"`, the actual output concept is supplied
+at run time via `pipe_run_params.dynamic_output_concept_ref` (or the legacy
+`params[DYNAMIC_OUTPUT_CONCEPT]` key). The resolver returns a fresh `StuffSpec` with
+the resolved concept, leaving `self.output` unchanged so the same pipe instance can
+be reused across runs with different overrides.
+
+Pinned behaviors:
+
+- explicit override (qualified ref or bare code) → returned StuffSpec carries that concept
+- no override → fallback `native.Text`
+- non-Dynamic output → returns `self.output` unchanged
+- `self.output.concept` is **never mutated** (regression guard for the cubic P1 bug
+ where the resolver mutated state, making the second run on the same instance
+ silently ignore a different `dynamic_output_concept_ref`)
+"""
+
+from typing import Callable
+
+from pipelex.core.concepts.native.concept_native import NativeConceptCode
+from pipelex.core.domains.domain import SpecialDomain
+from pipelex.core.pipes.pipe_factory import PipeFactory
+from pipelex.pipe_operators.llm.pipe_llm import PipeLLM
+from pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint
+from pipelex.pipe_run.pipe_run_params import PipeRunParamKey
+from pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactory
+
+
+def _make_pipe(domain_code: str, pipe_code: str, output: str) -> PipeLLM:
+ blueprint = PipeLLMBlueprint(
+ description="dynamic-output test pipe",
+ inputs={"user_text": "native.Text"},
+ output=output,
+ prompt="Process @user_text",
+ )
+ return PipeFactory[PipeLLM].make_from_blueprint(
+ domain_code=domain_code,
+ pipe_code=pipe_code,
+ blueprint=blueprint,
+ )
+
+
+class TestPipeLLMDynamicOutputResolution:
+ def test_fallback_to_native_text_when_no_override(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ """Regression: when the run params provide no override, the resolver still
+ returns a StuffSpec carrying `native.Text` (previously, the fallback was
+ only assigned to a local variable, so downstream code ran with `Dynamic`
+ and the LLM's structured JSON deserialized to `{}`).
+ """
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_fallback", output="native.Dynamic")
+ params = PipeRunParamsFactory.make_run_params() # no dynamic_output_concept_ref
+
+ resolved = pipe_llm.resolve_dynamic_output_stuff_spec(pipe_run_params=params)
+
+ assert resolved.concept.code == NativeConceptCode.TEXT
+ assert resolved.concept.domain_code == SpecialDomain.NATIVE
+ # self.output is untouched.
+ assert pipe_llm.output.concept.code == NativeConceptCode.DYNAMIC
+ assert pipe_llm.output.concept.domain_code == SpecialDomain.NATIVE
+
+ def test_override_with_qualified_native_ref(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_qualified", output="native.Dynamic")
+ params = PipeRunParamsFactory.make_run_params(dynamic_output_concept_ref="native.Number")
+
+ resolved = pipe_llm.resolve_dynamic_output_stuff_spec(pipe_run_params=params)
+
+ assert resolved.concept.code == NativeConceptCode.NUMBER
+ assert resolved.concept.domain_code == SpecialDomain.NATIVE
+ assert pipe_llm.output.concept.code == NativeConceptCode.DYNAMIC
+
+ def test_override_with_bare_code_uses_pipe_domain(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ """A bare concept code (no `domain.` prefix) is resolved against the pipe's own
+ domain — for native concepts that means `native.<Code>` is found in the library.
+ This guards against accidental double-prefixing (`test_domain.native.Number`).
+ """
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_bare", output="native.Dynamic")
+ params = PipeRunParamsFactory.make_run_params(dynamic_output_concept_ref="Number")
+
+ resolved = pipe_llm.resolve_dynamic_output_stuff_spec(pipe_run_params=params)
+
+ assert resolved.concept.code == NativeConceptCode.NUMBER
+ assert resolved.concept.domain_code == SpecialDomain.NATIVE
+
+ def test_legacy_params_key_override(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ """Legacy callers set the override via `params[_dynamic_output_concept]` instead
+ of the dedicated field. The resolver must honor it as a fallback so existing
+ integrations keep working.
+ """
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_legacy", output="native.Dynamic")
+ params = PipeRunParamsFactory.make_run_params(params={PipeRunParamKey.DYNAMIC_OUTPUT_CONCEPT: "native.Number"})
+
+ resolved = pipe_llm.resolve_dynamic_output_stuff_spec(pipe_run_params=params)
+
+ assert resolved.concept.code == NativeConceptCode.NUMBER
+ assert resolved.concept.domain_code == SpecialDomain.NATIVE
+
+ def test_non_dynamic_output_is_unchanged(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_non_dynamic", output="native.Text")
+ params = PipeRunParamsFactory.make_run_params(dynamic_output_concept_ref="native.Number")
+
+ resolved = pipe_llm.resolve_dynamic_output_stuff_spec(pipe_run_params=params)
+
+ # Non-Dynamic output: helper returns the static spec unchanged, ignoring the override.
+ assert resolved is pipe_llm.output
+ assert resolved.concept.code == NativeConceptCode.TEXT
+ assert resolved.concept.domain_code == SpecialDomain.NATIVE
+
+ def test_resolves_independently_across_invocations(
+ self,
+ load_empty_library: Callable[[], str],
+ ):
+ """Regression for the cubic P1: the same pipe instance must resolve a different
+ dynamic output concept on each call. The previous implementation mutated
+ `self.output.concept` on the first call, so the guard at the top of the helper
+ (which short-circuits when concept is no longer Dynamic) silently ignored every
+ subsequent override.
+ """
+ load_empty_library()
+ pipe_llm = _make_pipe(domain_code="test_domain", pipe_code="test_dyn_repeat", output="native.Dynamic")
+
+ first = pipe_llm.resolve_dynamic_output_stuff_spec(
+ pipe_run_params=PipeRunParamsFactory.make_run_params(dynamic_output_concept_ref="native.Number"),
+ )
+ assert first.concept.code == NativeConceptCode.NUMBER
+ # self.output stays Dynamic between runs.
+ assert pipe_llm.output.concept.code == NativeConceptCode.DYNAMIC
+
+ second = pipe_llm.resolve_dynamic_output_stuff_spec(
+ pipe_run_params=PipeRunParamsFactory.make_run_params(dynamic_output_concept_ref="native.Text"),
+ )
+ assert second.concept.code == NativeConceptCode.TEXT
+ # First call's resolution didn't leak into the second.
+ assert first.concept.code == NativeConceptCode.NUMBER
+ assert pipe_llm.output.concept.code == NativeConceptCode.DYNAMIC
diff --git a/uv.lock b/uv.lock
index 6167230d5..d18ed6bc5 100644
--- a/uv.lock
+++ b/uv.lock
@@ -2480,7 +2480,7 @@ wheels = [
[[package]]
name = "mthds"
-version = "0.2.0"
+version = "0.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "backports-strenum", marker = "python_full_version < '3.11'" },
@@ -2491,9 +2491,9 @@ dependencies = [
{ name = "tomlkit" },
{ name = "typing-extensions" },
]
-sdist = { url = "https://files.pythonhosted.org/packages/c6/e8/69d60b665f10a28c5efede63ccf95bdf75954701023e38138dc97da5862a/mthds-0.2.0.tar.gz", hash = "sha256:95952b489e0a6336667086a3eddb0f761f81d4af29f32de28b4182b33d265fc9", size = 112564, upload-time = "2026-03-24T13:33:51.927Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/5b/3c/eea615b49c011657246ff61371ca4a95ab3baa94d4b79541bd4b0f816ae5/mthds-0.3.0.tar.gz", hash = "sha256:de479f977b2ec3614b23bdb53ed3e28e2332a2f1653ee92e118c530fe3d5d820", size = 112693, upload-time = "2026-04-29T12:50:11.426Z" }
wheels = [
- { url = "https://files.pythonhosted.org/packages/cc/64/3fe70f33f0f9c6237336f368cf0a24d28fce60fc1d486f0df308b36d3beb/mthds-0.2.0-py3-none-any.whl", hash = "sha256:72f6765ef48caff24211c0fc3523f61a7118ae829bd54caf1b9eba7fb4e4f281", size = 49346, upload-time = "2026-03-24T13:33:50.479Z" },
+ { url = "https://files.pythonhosted.org/packages/f3/86/1184f31583d3089baea9316853b329720cb786d8e01f38f303a17f1e3f4c/mthds-0.3.0-py3-none-any.whl", hash = "sha256:3f9822c23beb82607ec4bafcff99717fb6531eed8d19b86086dc9deb7825b06b", size = 49356, upload-time = "2026-04-29T12:50:09.734Z" },
]
[[package]]
@@ -3645,7 +3645,7 @@ requires-dist = [
{ name = "mkdocs-meta-manager", marker = "extra == 'docs'", specifier = ">=1.1.0" },
{ name = "mkdocs-redirects", marker = "extra == 'docs'", specifier = "==1.2.2" },
{ name = "moto", extras = ["s3"], marker = "extra == 'dev'", specifier = ">=5.0.0" },
- { name = "mthds", specifier = ">=0.2.0" },
+ { name = "mthds", specifier = ">=0.3.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = "==1.19.1" },
{ name = "networkx", specifier = ">=3.4.2" },
{ name = "openai", specifier = ">=1.108.1" },