Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Refactor/epic renaming 1 pipe jinja2 (#296)

Louis Choquel77e12933fb41
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c51115498..ca2cbc952 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -219,7 +219,7 @@ For complete details, see the [Inference Backend Configuration](pages/configurat
- Refactored the concepts: Blueprints are now more explicit, and hold only concept strings or code. Pipes hold concept instances.
- Organized code: Created subfolders for controller and operator pipes.
- Say goodbye to `PipeLLMPrompt`.
-- Removed the `PipeJinja2` and `PipeLLMPrompt` from the `PipeLLM`.
+- Removed the `PipeCompose` and `PipeLLMPrompt` from the `PipeLLM`.
### Added
@@ -314,7 +314,7 @@ Simplified input memory:
- Refactored `PipeInputSpec` to use `InputRequirement` and `TypedNamedInputRequirement` classes instead of plain strings for input specifications.
- Updated `WorkingMemoryFactory` to handle `ImplicitMemory` instead of `CompactMemory`.
- Replaced `ExecutePipelineException` with `PipelineInputError` in `execute_pipeline` function.
-- Updated `PipeBatch`, `PipeCondition`, `PipeParallel`, `PipeSequence`, `PipeFunc`, `PipeImgGen`, `PipeJinja2`, `PipeLLM`, and `PipeOcr` classes to use `InputRequirement` for input handling.
+- Updated `PipeBatch`, `PipeCondition`, `PipeParallel`, `PipeSequence`, `PipeFunc`, `PipeImgGen`, `PipeCompose`, `PipeLLM`, and `PipeOcr` classes to use `InputRequirement` for input handling.
- Updated `PipeInputSpec` creation in various test files to use `make_from_dict` method.
- Updated `pyproject.toml` to exclude `pypdfium2` version `4.30.1`.
- Updated `Jinja2TemplateCategory` to handle HTML and Markdown templates differently.
diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeJinja2.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeJinja2.md
index 56e16d569..228c37e7e 100644
--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeJinja2.md
+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeJinja2.md
@@ -1,10 +1,10 @@
-# PipeJinja2
+# PipeCompose
-The `PipeJinja2` operator is a powerful utility for rendering [Jinja2 templates](https://jinja.palletsprojects.com/). It allows you to dynamically generate text by combining data from your pipeline's working memory with a template. This is ideal for creating formatted reports, HTML content, or constructing complex, multi-part prompts for LLMs.
+The `PipeCompose` operator is a powerful utility for rendering [Jinja2 templates](https://jinja.palletsprojects.com/). It allows you to dynamically generate text by combining data from your pipeline's working memory with a template. This is ideal for creating formatted reports, HTML content, or constructing complex, multi-part prompts for LLMs.
## How it works
-`PipeJinja2` takes all the data currently in the `WorkingMemory` and uses it as the context for rendering a Jinja2 template. The resulting text is then saved back to the working memory as a new `Text` output.
+`PipeCompose` takes all the data currently in the `WorkingMemory` and uses it as the context for rendering a Jinja2 template. The resulting text is then saved back to the working memory as a new `Text` output.
The template itself can be provided in one of two ways:
1. **By Name**: Referring to a template file that has been loaded into Pipelex's template provider. This is the most common and maintainable method.
@@ -16,13 +16,13 @@ The Jinja2 template has access to all the "stuffs" currently in the working memo
## Configuration
-`PipeJinja2` is configured in your pipeline's `.plx` file.
+`PipeCompose` is configured in your pipeline's `.plx` file.
### PLX Parameters
| Parameter | Type | Description | Required |
| --------------- | --------------- | --------------------------------------------------------------------------------------------------------- | --------------------------- |
-| `type` | string | The type of the pipe: `PipeJinja2` | Yes |
+| `type` | string | The type of the pipe: `PipeCompose` | Yes |
| `definition` | string | A description of the Jinja2 operation. | Yes |
| `output` | string | The concept for the output. Defaults to `native.Text`. | No |
| `jinja2_name` | string | The name of a pre-loaded template file. | Yes (or `jinja2`) |
@@ -53,7 +53,7 @@ Report generated on: {{ report_date }}
**Pipeline PLX definition:**
```plx
[pipe.generate_weekly_report]
-type = "PipeJinja2"
+type = "PipeCompose"
definition = "Generate a formatted weekly report for a user"
output = "WeeklyReportText"
jinja2_name = "weekly_report.md"
@@ -61,7 +61,7 @@ extra_context = { report_date = "2023-10-27" }
```
In this scenario:
-- `PipeJinja2` will load the `weekly_report.md` template.
+- `PipeCompose` will load the `weekly_report.md` template.
- It will use the `user` and `activities` objects from the working memory.
- It will add `report_date` to the context from the `extra_context` table.
- The rendered markdown text will be saved as the `WeeklyReportText` concept.
\ No newline at end of file
diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/index.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/index.md
index 8b209edf5..7172bc539 100644
--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/index.md
+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/index.md
@@ -12,7 +12,7 @@ Here are the primary pipe operators available in Pipelex:
- [**`PipeOcr`**](./PipeOcr.md): Performs Optical Character Recognition (OCR) on images and PDF documents to extract text and embedded images.
- [**`PipeImgGen`**](./PipeImgGen.md): Generates images from a text prompt using models like GPT Image, Flux, or other image generation models.
- [**`PipeFunc`**](./PipeFunc.md): An escape hatch that allows you to execute any custom Python function, giving you maximum flexibility.
-- [**`PipeJinja2`**](./PipeJinja2.md): Renders a Jinja2 template using data from the working memory, perfect for creating formatted reports or complex prompts.
+- [**`PipeCompose`**](./PipeCompose.md): Renders a Jinja2 template using data from the working memory, perfect for creating formatted reports or complex prompts.
## Overview
diff --git a/mkdocs.yml b/mkdocs.yml
index 42c129323..da1c6d7aa 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -115,7 +115,7 @@ nav:
- PipeLLM: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md
- PipeOcr: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeOcr.md
- PipeImgGen: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeImgGen.md
- - PipeJinja2: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeJinja2.md
+ - PipeCompose: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeCompose.md
- PipeFunc: pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeFunc.md
- Pipe Controllers:
- Overview: pages/build-reliable-ai-workflows-with-pipelex/pipe-controllers/index.md
diff --git a/pipelex/cogt/llm/llm_prompt_spec.py b/pipelex/cogt/llm/llm_prompt_spec.py
index 59da2e258..32457def0 100644
--- a/pipelex/cogt/llm/llm_prompt_spec.py
+++ b/pipelex/cogt/llm/llm_prompt_spec.py
@@ -129,8 +129,9 @@ class LLMPromptSpec(BaseModel):
extra_params=extra_params,
)
if not user_text:
- msg = "For user_text we need either a pipe_jinja2, a text_verbatim_name or a fixed user_text"
- raise ValueError(msg)
+ # TODO: link to docs
+ msg = "Could not unravel user_text, we need either a jinja2_blueprint, a text_verbatim_name or a fixed user_text"
+ raise LLMPromptSpecError(msg)
if output_structure_prompt:
user_text += output_structure_prompt
@@ -191,7 +192,7 @@ class LLMPromptSpec(BaseModel):
)
if not user_text_verbatim:
msg = f"Could not find text_verbatim template '{text_verbatim_name}'"
- raise ValueError(msg)
+ raise LLMPromptSpecError(msg)
the_text = user_text_verbatim
elif fixed_text:
the_text = fixed_text
diff --git a/pipelex/core/bundles/pipelex_bundle_blueprint.py b/pipelex/core/bundles/pipelex_bundle_blueprint.py
index 91cb86c39..d5f604ab8 100644
--- a/pipelex/core/bundles/pipelex_bundle_blueprint.py
+++ b/pipelex/core/bundles/pipelex_bundle_blueprint.py
@@ -8,16 +8,16 @@ from pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBluepri
from pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeConditionBlueprint
from pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprint
from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
from pipelex.pipe_operators.func.pipe_func_blueprint import PipeFuncBlueprint
from pipelex.pipe_operators.img_gen.pipe_img_gen_blueprint import PipeImgGenBlueprint
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
from pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint
from pipelex.pipe_operators.ocr.pipe_ocr_blueprint import PipeOcrBlueprint
PipeBlueprintUnion = Annotated[
PipeFuncBlueprint
| PipeImgGenBlueprint
- | PipeJinja2Blueprint
+ | PipeComposeBlueprint
| PipeLLMBlueprint
| PipeOcrBlueprint
| PipeBatchBlueprint
diff --git a/pipelex/core/pipes/pipe_blueprint.py b/pipelex/core/pipes/pipe_blueprint.py
index 07e1c5fe8..fc3dbe731 100644
--- a/pipelex/core/pipes/pipe_blueprint.py
+++ b/pipelex/core/pipes/pipe_blueprint.py
@@ -22,7 +22,7 @@ class AllowedPipeTypes(StrEnum):
# Pipe Operators
PIPE_FUNC = "PipeFunc"
PIPE_IMG_GEN = "PipeImgGen"
- PIPE_JINJA2 = "PipeJinja2"
+ PIPE_COMPOSE = "PipeCompose"
PIPE_LLM = "PipeLLM"
PIPE_OCR = "PipeOcr"
# Pipe Controller
diff --git a/pipelex/core/registry_models.py b/pipelex/core/registry_models.py
index 644b8795b..69cecba2e 100644
--- a/pipelex/core/registry_models.py
+++ b/pipelex/core/registry_models.py
@@ -24,12 +24,12 @@ from pipelex.pipe_controllers.parallel.pipe_parallel import PipeParallel
from pipelex.pipe_controllers.parallel.pipe_parallel_factory import PipeParallelFactory
from pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequence
from pipelex.pipe_controllers.sequence.pipe_sequence_factory import PipeSequenceFactory
+from pipelex.pipe_operators.compose.pipe_compose import PipeCompose
+from pipelex.pipe_operators.compose.pipe_compose_factory import PipeComposeFactory
from pipelex.pipe_operators.func.pipe_func import PipeFunc
from pipelex.pipe_operators.func.pipe_func_factory import PipeFuncFactory
from pipelex.pipe_operators.img_gen.pipe_img_gen import PipeImgGen
from pipelex.pipe_operators.img_gen.pipe_img_gen_factory import PipeImgGenFactory
-from pipelex.pipe_operators.jinja2.pipe_jinja2 import PipeJinja2
-from pipelex.pipe_operators.jinja2.pipe_jinja2_factory import PipeJinja2Factory
from pipelex.pipe_operators.llm.pipe_llm import PipeLLM
from pipelex.pipe_operators.llm.pipe_llm_factory import PipeLLMFactory
from pipelex.pipe_operators.ocr.pipe_ocr import PipeOcr
@@ -43,7 +43,7 @@ class PipelexRegistryModels(RegistryModels):
PIPE_OPERATORS: ClassVar[list[PipeAbstractType]] = [
PipeFunc,
PipeImgGen,
- PipeJinja2,
+ PipeCompose,
PipeLLM,
PipeOcr,
]
@@ -51,7 +51,7 @@ class PipelexRegistryModels(RegistryModels):
PIPE_OPERATORS_FACTORY: ClassVar[list[PipeFactoryProtocol[Any, Any]]] = [
PipeFuncFactory,
PipeImgGenFactory,
- PipeJinja2Factory,
+ PipeComposeFactory,
PipeLLMFactory,
PipeOcrFactory,
]
diff --git a/pipelex/libraries/pipelines/builder/builder.plx b/pipelex/libraries/pipelines/builder/builder.plx
index 2289a482e..735c95ae1 100644
--- a/pipelex/libraries/pipelines/builder/builder.plx
+++ b/pipelex/libraries/pipelines/builder/builder.plx
@@ -232,7 +232,7 @@ So the input of PipeOcr is {ocr_input: "PDF"} or {ocr_input: "Image"} or a conce
**PipeFunc**: A pipe that executes a custom Python function.
- function_name: Name of the Python function to call
-**PipeJinja2**: A pipe that uses Jinja2 to render a template.
+**PipeCompose**: A pipe that uses Jinja2 to render a template.
- jinja2: Raw Jinja2 template string OR
- jinja2_name: Name reference to a template (use one or the other)
@@ -326,7 +326,7 @@ steps = [
]
[pipe.check_validation_status]
-type = "PipeJinja2"
+type = "PipeCompose"
description = "Check if validation failed by examining if failed_pipes list is empty."
inputs = { failed_pipes = "PipeFailure" }
output = "Text"
@@ -362,7 +362,7 @@ output = "PipeFailure"
function_name = "validate_dry_run"
[pipe.continue]
-type = "PipeJinja2"
+type = "PipeCompose"
description = "Continue with successful validation - return the bundle unchanged."
inputs = { pipelex_bundle_spec = "PipelexBundleSpec" }
output = "PipelexBundleSpec"
diff --git a/pipelex/libraries/pipelines/builder/builder.py b/pipelex/libraries/pipelines/builder/builder.py
index e3ad8d3fe..4493cf68d 100644
--- a/pipelex/libraries/pipelines/builder/builder.py
+++ b/pipelex/libraries/pipelines/builder/builder.py
@@ -10,10 +10,10 @@ from pipelex.core.stuffs.stuff_content import ListContent, StructuredContent
from pipelex.hub import get_library_manager
from pipelex.libraries.pipelines.builder.concept.concept_spec import ConceptSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_batch_spec import PipeBatchSpec
+from pipelex.libraries.pipelines.builder.pipe.pipe_compose_spec import PipeComposeSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_condition_spec import PipeConditionSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_func_spec import PipeFuncSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_img_spec import PipeImgGenSpec
-from pipelex.libraries.pipelines.builder.pipe.pipe_jinja2_spec import PipeJinja2Spec
from pipelex.libraries.pipelines.builder.pipe.pipe_llm_spec import PipeLLMSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_ocr_spec import PipeOcrSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_parallel_spec import PipeParallelSpec
@@ -47,7 +47,7 @@ class PipelexBundleSpecDraft(StructuredContent):
PipeSpecUnion = Annotated[
PipeFuncSpec
| PipeImgGenSpec
- | PipeJinja2Spec
+ | PipeComposeSpec
| PipeLLMSpec
| PipeOcrSpec
| PipeBatchSpec
@@ -142,7 +142,7 @@ def _convert_pipe_spec(pipe_spec: PipeSpecUnion) -> PipeSpecUnion:
pipe_type_to_class: dict[str, type] = {
"PipeFunc": PipeFuncSpec,
"PipeImgGen": PipeImgGenSpec,
- "PipeJinja2": PipeJinja2Spec,
+ "PipeCompose": PipeComposeSpec,
"PipeLLM": PipeLLMSpec,
"PipeOcr": PipeOcrSpec,
"PipeBatch": PipeBatchSpec,
@@ -238,7 +238,7 @@ async def validate_dry_run(working_memory: WorkingMemory) -> ListContent[PipeFai
pipe_type_to_spec_class = {
"PipeFunc": PipeFuncSpec,
"PipeImgGen": PipeImgGenSpec,
- "PipeJinja2": PipeJinja2Spec,
+ "PipeCompose": PipeComposeSpec,
"PipeLLM": PipeLLMSpec,
"PipeOcr": PipeOcrSpec,
"PipeBatch": PipeBatchSpec,
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe.plx b/pipelex/libraries/pipelines/builder/pipe/pipe.plx
index 8c9f8da62..8e754a15b 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe.plx
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe.plx
@@ -12,7 +12,7 @@ PipeSequenceSpec = "A structured spec for a pipe sequence."
# Pipe operators
PipeFuncSpec = "A structured spec for a pipe func."
PipeImgGenSpec = "A structured spec for a pipe img gen."
-PipeJinja2Spec = "A structured spec for a pipe jinja2."
+PipeComposeSpec = "A structured spec for a pipe compose."
PipeLLMSpec = "A structured spec for a pipe llm."
PipeOcrSpec = "A structured spec for a pipe ocr."
PipeFailure = "Details of a single pipe failure during dry run."
@@ -68,7 +68,7 @@ PipeBatch = "emit_batch_from_signature"
PipeLLM = "emit_llm_from_signature"
PipeOcr = "emit_ocr_from_signature"
PipeImgGen = "emit_imggen_from_signature"
-PipeJinja2 = "emit_jinja_from_signature"
+PipeCompose = "emit_compose_from_signature"
PipeFunc = "emit_func_from_signature"
# ────────────────────────────────────────────────────────────────────────────────
@@ -224,13 +224,13 @@ And here are the concepts you can use:
@concept_specs
"""
-[pipe.emit_jinja_from_signature]
+[pipe.emit_compose_from_signature]
type = "PipeLLM"
-description = "Build a PipeJinja2Spec from the signature."
+description = "Build a PipeComposeSpec from the signature."
inputs = { pipe_signature = "PipeSignature", concept_specs = "concept.ConceptSpec", pipe_spec = "PipeSpec" }
-output = "PipeJinja2Spec"
+output = "PipeComposeSpec"
prompt_template = """
-Return a PipeJinja2Spec for this signature.
+Return a PipeComposeSpec for this signature.
Signature:
@pipe_signature
@@ -277,7 +277,7 @@ PipeLLM = "fix_failing_llm_pipe"
PipeImgGen = "fix_failing_imggen_pipe"
PipeOcr = "fix_failing_ocr_pipe"
PipeFunc = "fix_failing_func_pipe"
-PipeJinja2 = "fix_failing_jinja2_pipe"
+PipeCompose = "fix_failing_compose_pipe"
PipeSequence = "fix_failing_sequence_pipe"
PipeParallel = "fix_failing_parallel_pipe"
PipeCondition = "fix_failing_condition_pipe"
@@ -371,14 +371,14 @@ Please provide only the corrected PipeFuncSpec. Common Func pipe issues to fix:
- Function not available in registry
"""
-[pipe.fix_failing_jinja2_pipe]
+[pipe.fix_failing_compose_pipe]
type = "PipeLLM"
-description = "Fix a failing PipeJinja2 spec based on its specific error."
+description = "Fix a failing PipeCompose spec based on its specific error."
inputs = { pipelex_bundle_spec = "PipelexBundleSpec", failed_pipe = "PipeFailure" }
-output = "PipeJinja2Spec"
+output = "PipeComposeSpec"
llm = "llm_to_engineer"
prompt_template = """
-Fix this failing PipeJinja2 spec.
+Fix this failing PipeCompose spec.
Failing pipe:
@failed_pipe.pipe
@@ -386,7 +386,7 @@ Failing pipe:
Error message:
@failed_pipe.error_message
-Please provide only the corrected PipeJinja2Spec. Common Jinja2 pipe issues to fix:
+Please provide only the corrected PipeComposeSpec. Common compose pipe issues to fix:
- Invalid Jinja2 template syntax
- Missing input variables referenced in template
- Wrong concept types for inputs/outputs
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe_jinja2_spec.py b/pipelex/libraries/pipelines/builder/pipe/pipe_compose_spec.py
similarity index 88%
rename from pipelex/libraries/pipelines/builder/pipe/pipe_jinja2_spec.py
rename to pipelex/libraries/pipelines/builder/pipe/pipe_compose_spec.py
index 83a9d1cc8..7555c7457 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe_jinja2_spec.py
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe_compose_spec.py
@@ -5,7 +5,7 @@ from typing_extensions import override
from pipelex.core.stuffs.stuff_content import StructuredContent
from pipelex.libraries.pipelines.builder.pipe.pipe_signature import PipeSpec
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
from pipelex.tools.templating.jinja2_template_category import Jinja2TemplateCategory
from pipelex.tools.templating.templating_models import PromptingStyle, TagStyle, TextFormat
@@ -15,7 +15,10 @@ class PromptingStyleSpec(StructuredContent):
text_format: TextFormat = Field(TextFormat.PLAIN, strict=False)
-class Jinja2Spec(StructuredContent):
+class PipeComposeSpec(PipeSpec):
+ type: Literal["PipeCompose"] = "PipeCompose"
+ category: Literal["PipeOperator"] = "PipeOperator"
+ the_pipe_code: str = Field(description="Pipe code. Must be snake_case.")
jinja2_name: str | None = Field(default=None, description="Name of the Jinja2 template to use")
jinja2: str | None = Field(default=None, description="Raw Jinja2 template string")
prompting_style: PromptingStyleSpec | None = Field(default=None, description="Style of prompting to use (typically for different LLMs)")
@@ -25,14 +28,8 @@ class Jinja2Spec(StructuredContent):
)
extra_context: dict[str, Any] | None = Field(default=None, description="Additional context variables for template rendering")
-
-class PipeJinja2Spec(PipeSpec, Jinja2Spec):
- type: Literal["PipeJinja2"] = "PipeJinja2"
- category: Literal["PipeOperator"] = "PipeOperator"
- the_pipe_code: str = Field(description="Pipe code. Must be snake_case.")
-
@override
- def to_blueprint(self) -> PipeJinja2Blueprint:
+ def to_blueprint(self) -> PipeComposeBlueprint:
base_blueprint = super().to_blueprint()
if self.prompting_style:
@@ -44,7 +41,7 @@ class PipeJinja2Spec(PipeSpec, Jinja2Spec):
else:
prompting_style = None
- return PipeJinja2Blueprint(
+ return PipeComposeBlueprint(
definition=base_blueprint.definition,
inputs=base_blueprint.inputs,
output=base_blueprint.output,
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py b/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
index 90a5132ca..8c9f667a5 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
@@ -40,7 +40,7 @@ class PipeSpec(StructuredContent):
category: The pipe category (PipeOperator, PipeController). Uses Any type to avoid
category override conflicts but validated at runtime.
The pipe controllers are PipeSequence, PipeParallel, PipeCondition, PipeBatch.
- The pipe operators are PipeFunc, PipeLLM, PipeImgGen, PipeOcr, PipeJinja2.
+ The pipe operators are PipeFunc, PipeLLM, PipeImgGen, PipeOcr, PipeCompose.
definition: Natural language description of what the pipe does.
inputs: Input concept specifications. should be an InputRequirementBlueprint
Dictionary keys are input names in snake_case, values are concept specifications in PascalCase.
diff --git a/pipelex/migration/migrate_v0_1_0_to_v0_2_0.py b/pipelex/migration/migrate_v0_1_0_to_v0_2_0.py
index e9ec77e70..702d4c8c3 100644
--- a/pipelex/migration/migrate_v0_1_0_to_v0_2_0.py
+++ b/pipelex/migration/migrate_v0_1_0_to_v0_2_0.py
@@ -18,7 +18,7 @@ class TOMLMigrator:
"PipeOcr",
"PipeImgGen",
"PipeFunc",
- "PipeJinja2",
+ "PipeCompose",
"PipeLLMPrompt",
"PipeSequence",
"PipeBatch",
diff --git a/pipelex/pipe_controllers/condition/pipe_condition.py b/pipelex/pipe_controllers/condition/pipe_condition.py
index 793a84c16..7905e8c35 100644
--- a/pipelex/pipe_controllers/condition/pipe_condition.py
+++ b/pipelex/pipe_controllers/condition/pipe_condition.py
@@ -1,4 +1,4 @@
-from typing import TYPE_CHECKING, Literal, cast
+from typing import Literal
import shortuuid
from pydantic import field_validator, model_validator
@@ -28,17 +28,13 @@ from pipelex.exceptions import (
from pipelex.hub import get_pipe_router, get_pipeline_tracker, get_required_pipe
from pipelex.pipe_controllers.condition.pipe_condition_details import PipeConditionDetails, PipeConditionPipeMap
from pipelex.pipe_controllers.pipe_controller import PipeController
-from pipelex.pipe_operators.jinja2.pipe_jinja2 import PipeJinja2Output
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
-from pipelex.pipe_operators.jinja2.pipe_jinja2_factory import PipeJinja2Factory
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
+from pipelex.pipe_operators.compose.pipe_compose_factory import PipeComposeFactory
from pipelex.pipe_works.pipe_job_factory import PipeJobFactory
from pipelex.pipeline.job_metadata import JobCategory, JobMetadata
from pipelex.tools.typing.validation_utils import has_exactly_one_among_attributes_from_list
from pipelex.types import Self
-if TYPE_CHECKING:
- from pipelex.pipe_operators.jinja2.pipe_jinja2 import PipeJinja2Output
-
class PipeCondition(PipeController):
type: Literal["PipeCondition"] = "PipeCondition"
@@ -130,13 +126,14 @@ class PipeCondition(PipeController):
@override
def required_variables(self) -> set[str]:
required_variables: set[str] = set()
+ # TODO: use jinja2 directly without going though a pipe
# Variables from the expression/expression_template
- pipe_jinja2 = PipeJinja2Factory.make_pipe_jinja2_from_template_str(
+ pipe_compose = PipeComposeFactory.make_pipe_compose_from_template_str(
domain=self.domain,
template_str=self.applied_expression_template,
inputs=self.inputs,
)
- required_variables.update(pipe_jinja2.required_variables())
+ required_variables.update(pipe_compose.required_variables())
# Variables from the pipe_map
for pipe_code in self.pipe_dependencies():
@@ -165,13 +162,13 @@ class PipeCondition(PipeController):
needed_inputs = PipeInputSpecFactory.make_empty()
# 1. Add the variables from the expression/expression_template
- pipe_jinja2 = PipeJinja2Factory.make_pipe_jinja2_from_template_str(
+ pipe_compose = PipeComposeFactory.make_pipe_compose_from_template_str(
domain=self.domain,
template_str=self.applied_expression_template,
inputs=self.inputs,
)
- for var_name in pipe_jinja2.required_variables():
+ for var_name in pipe_compose.required_variables():
if not var_name.startswith("_"): # exclude internal variables starting with `_`
# We don't know the concept code from just the variable name,
# so we'll use a generic placeholder that will be validated later
@@ -285,7 +282,7 @@ class PipeCondition(PipeController):
multiplicity=requirement.multiplicity,
)
- pipe_jinja2_blueprint = PipeJinja2Blueprint(
+ pipe_compose_blueprint = PipeComposeBlueprint(
definition="Jinja2 template for pipe condition evaluation",
jinja2=self.applied_expression_template,
inputs=inputs_blueprint,
@@ -293,10 +290,10 @@ class PipeCondition(PipeController):
)
# TODO: use jinja2 directly without going though a pipe
- pipe_jinja2 = PipeJinja2Factory.make_from_blueprint(
+ pipe_compose = PipeComposeFactory.make_from_blueprint(
domain=self.domain,
pipe_code="evaluation_for_pipe_condition",
- blueprint=pipe_jinja2_blueprint,
+ blueprint=pipe_compose_blueprint,
)
jinja2_job_metadata = job_metadata.copy_with_update(
updated_metadata=JobMetadata(
@@ -304,23 +301,13 @@ class PipeCondition(PipeController):
),
)
log.debug(f"Jinja2 expression: {self.applied_expression_template}")
- # evaluated_expression = (
- # await pipe_jinja2.run_pipe(
- # job_metadata=jinja2_job_metadata,
- # working_memory=working_memory,
- # pipe_run_params=pipe_run_params,
- # )
- # ).rendered_text.strip()
- # TODO: restore the possibility above, without need to explicitly cast the output
- pipe_jinja2_output = cast(
- "PipeJinja2Output",
- await pipe_jinja2.run_pipe(
+ evaluated_expression = (
+ await pipe_compose.run_pipe(
job_metadata=jinja2_job_metadata,
working_memory=working_memory,
pipe_run_params=pipe_run_params,
- ),
- )
- evaluated_expression = pipe_jinja2_output.rendered_text.strip()
+ )
+ ).main_stuff_as_str.strip()
if not evaluated_expression or evaluated_expression == "None":
error_msg = f"Conditional expression returned an empty string in pipe {self.code}:"
@@ -356,7 +343,7 @@ class PipeCondition(PipeController):
evaluated_expression=evaluated_expression,
chosen_pipe_code=chosen_pipe_code,
)
- required_variables = pipe_jinja2.required_variables()
+ required_variables = pipe_compose.required_variables()
log.debug(required_variables, title=f"Required variables for PipeCondition '{self.code}'")
required_stuff_names = {required_variable for required_variable in required_variables if not required_variable.startswith("_")}
try:
@@ -426,13 +413,13 @@ class PipeCondition(PipeController):
# 2. Validate that the expression template is valid
try:
- pipe_jinja2 = PipeJinja2Factory.make_pipe_jinja2_from_template_str(
+ pipe_compose = PipeComposeFactory.make_pipe_compose_from_template_str(
domain=self.domain,
template_str=self.applied_expression_template,
inputs=self.inputs,
)
# Get required variables to validate the template syntax
- required_variables = pipe_jinja2.required_variables()
+ required_variables = pipe_compose.required_variables()
log.debug(f"Expression template is valid, requires variables: {required_variables}")
except Exception as exc:
log.error(f"Dry run failed: invalid expression template: {exc}")
diff --git a/pipelex/pipe_operators/jinja2/__init__.py b/pipelex/pipe_operators/compose/__init__.py
similarity index 100%
rename from pipelex/pipe_operators/jinja2/__init__.py
rename to pipelex/pipe_operators/compose/__init__.py
diff --git a/pipelex/pipe_operators/jinja2/pipe_jinja2.py b/pipelex/pipe_operators/compose/pipe_compose.py
similarity index 90%
rename from pipelex/pipe_operators/jinja2/pipe_jinja2.py
rename to pipelex/pipe_operators/compose/pipe_compose.py
index 342b4a74d..7d5399fe6 100644
--- a/pipelex/pipe_operators/jinja2/pipe_jinja2.py
+++ b/pipelex/pipe_operators/compose/pipe_compose.py
@@ -32,14 +32,12 @@ from pipelex.tools.typing.validation_utils import has_exactly_one_among_attribut
from pipelex.types import Self
-class PipeJinja2Output(PipeOutput):
- @property
- def rendered_text(self) -> str:
- return self.main_stuff_as_text.text
+class PipeComposeOutput(PipeOutput):
+ pass
-class PipeJinja2(PipeOperator[PipeJinja2Output]):
- type: Literal["PipeJinja2"] = "PipeJinja2"
+class PipeCompose(PipeOperator[PipeComposeOutput]):
+ type: Literal["PipeCompose"] = "PipeCompose"
model_config = ConfigDict(extra="forbid", strict=False)
adhoc_pipe_code: ClassVar[str] = "jinja2_render"
@@ -56,13 +54,13 @@ class PipeJinja2(PipeOperator[PipeJinja2Output]):
@model_validator(mode="after")
def validate_jinja2(self) -> Self:
if not has_exactly_one_among_attributes_from_list(self, attributes_list=["jinja2_name", "jinja2"]):
- msg = "PipeJinja2 should have exactly one of jinja2_name or jinja2"
+ msg = "PipeCompose should have exactly one of jinja2_name or jinja2"
raise PipeDefinitionError(msg)
if self.jinja2:
try:
check_jinja2_parsing(jinja2_template_source=self.jinja2, template_category=self.template_category)
except TemplateSyntaxError as exc:
- msg = f"Could not parse Jinja2 template included in PipeJinja2: {exc}"
+ msg = f"Could not parse Jinja2 template included in PipeCompose: {exc}"
raise Jinja2TemplateError(msg) from exc
return self
@@ -128,10 +126,10 @@ class PipeJinja2(PipeOperator[PipeJinja2Output]):
pipe_run_params: PipeRunParams,
output_name: str | None = None,
content_generator: ContentGeneratorProtocol | None = None,
- ) -> PipeJinja2Output:
+ ) -> PipeComposeOutput:
content_generator = content_generator or get_content_generator()
if pipe_run_params.is_multiple_output_required:
- msg = f"PipeJinja2 does not suppport multiple outputs, got output_multiplicity = {pipe_run_params.output_multiplicity}"
+ msg = f"PipeCompose does not suppport multiple outputs, got output_multiplicity = {pipe_run_params.output_multiplicity}"
raise PipeRunParamsError(msg)
context: dict[str, Any] = working_memory.generate_context()
@@ -158,7 +156,7 @@ class PipeJinja2(PipeOperator[PipeJinja2Output]):
name=output_name,
)
- return PipeJinja2Output(
+ return PipeComposeOutput(
working_memory=working_memory,
pipeline_run_id=job_metadata.pipeline_run_id,
)
@@ -170,13 +168,13 @@ class PipeJinja2(PipeOperator[PipeJinja2Output]):
working_memory: WorkingMemory,
pipe_run_params: PipeRunParams,
output_name: str | None = None,
- ) -> PipeJinja2Output:
+ ) -> PipeComposeOutput:
content_generator_used: ContentGeneratorProtocol
if get_config().pipelex.dry_run_config.apply_to_jinja2_rendering:
- log.debug(f"PipeJinja2: using dry run operator pipe for jinja2 rendering: {self.code}")
+ log.debug(f"PipeCompose: using dry run operator pipe for jinja2 rendering: {self.code}")
content_generator_used = ContentGeneratorDry()
else:
- log.debug(f"PipeJinja2: using regular operator pipe for jinja2 rendering (dry run not applied to jinja2): {self.code}")
+ log.debug(f"PipeCompose: using regular operator pipe for jinja2 rendering (dry run not applied to jinja2): {self.code}")
content_generator_used = get_content_generator()
return await self._run_operator_pipe(
diff --git a/pipelex/pipe_operators/jinja2/pipe_jinja2_blueprint.py b/pipelex/pipe_operators/compose/pipe_compose_blueprint.py
similarity index 66%
rename from pipelex/pipe_operators/jinja2/pipe_jinja2_blueprint.py
rename to pipelex/pipe_operators/compose/pipe_compose_blueprint.py
index 0fefced7c..ac9422040 100644
--- a/pipelex/pipe_operators/jinja2/pipe_jinja2_blueprint.py
+++ b/pipelex/pipe_operators/compose/pipe_compose_blueprint.py
@@ -4,6 +4,6 @@ from pipelex.core.pipes.pipe_blueprint import PipeBlueprint
from pipelex.tools.templating.jinja2_blueprint import Jinja2Blueprint
-class PipeJinja2Blueprint(PipeBlueprint, Jinja2Blueprint):
- type: Literal["PipeJinja2"] = "PipeJinja2"
+class PipeComposeBlueprint(PipeBlueprint, Jinja2Blueprint):
+ type: Literal["PipeCompose"] = "PipeCompose"
category: Literal["PipeOperator"] = "PipeOperator"
diff --git a/pipelex/pipe_operators/jinja2/pipe_jinja2_factory.py b/pipelex/pipe_operators/compose/pipe_compose_factory.py
similarity index 80%
rename from pipelex/pipe_operators/jinja2/pipe_jinja2_factory.py
rename to pipelex/pipe_operators/compose/pipe_compose_factory.py
index 2ce9d7436..4dc329138 100644
--- a/pipelex/pipe_operators/jinja2/pipe_jinja2_factory.py
+++ b/pipelex/pipe_operators/compose/pipe_compose_factory.py
@@ -6,23 +6,23 @@ from pipelex.core.pipes.pipe_input import PipeInputSpec
from pipelex.core.pipes.pipe_input_factory import PipeInputSpecFactory
from pipelex.exceptions import PipeDefinitionError
from pipelex.hub import get_concept_provider
-from pipelex.pipe_operators.jinja2.pipe_jinja2 import PipeJinja2
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
+from pipelex.pipe_operators.compose.pipe_compose import PipeCompose
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
from pipelex.tools.templating.jinja2_parsing import check_jinja2_parsing
from pipelex.tools.templating.jinja2_template_category import Jinja2TemplateCategory
from pipelex.tools.templating.template_preprocessor import preprocess_template
-class PipeJinja2Factory(PipeFactoryProtocol[PipeJinja2Blueprint, PipeJinja2]):
+class PipeComposeFactory(PipeFactoryProtocol[PipeComposeBlueprint, PipeCompose]):
@classmethod
@override
def make_from_blueprint(
cls,
domain: str,
pipe_code: str,
- blueprint: PipeJinja2Blueprint,
+ blueprint: PipeComposeBlueprint,
concept_codes_from_the_same_domain: list[str] | None = None,
- ) -> PipeJinja2:
+ ) -> PipeCompose:
preprocessed_template: str | None = None
if blueprint.jinja2:
preprocessed_template = preprocess_template(blueprint.jinja2)
@@ -38,7 +38,7 @@ class PipeJinja2Factory(PipeFactoryProtocol[PipeJinja2Blueprint, PipeJinja2]):
concept_string_or_code=blueprint.output,
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
)
- return PipeJinja2(
+ return PipeCompose(
domain=domain,
code=pipe_code,
definition=blueprint.definition,
@@ -61,31 +61,32 @@ class PipeJinja2Factory(PipeFactoryProtocol[PipeJinja2Blueprint, PipeJinja2]):
)
@classmethod
- def make_pipe_jinja2_from_template_str(
+ def make_pipe_compose_from_template_str(
cls,
domain: str,
inputs: PipeInputSpec | None = None,
template_str: str | None = None,
template_name: str | None = None,
- ) -> PipeJinja2:
+ ) -> PipeCompose:
if template_str:
preprocessed_template = preprocess_template(template_str)
check_jinja2_parsing(
jinja2_template_source=preprocessed_template,
template_category=Jinja2TemplateCategory.LLM_PROMPT,
)
- return PipeJinja2(
+ return PipeCompose(
domain=domain,
- code="adhoc_pipe_jinja2_from_template_str",
+ code="adhoc_pipe_compose_from_template_str",
jinja2=preprocessed_template,
inputs=inputs or PipeInputSpecFactory.make_empty(),
)
- if template_name:
- return PipeJinja2(
+ elif template_name:
+ return PipeCompose(
domain=domain,
- code="adhoc_pipe_jinja2_from_template_name",
+ code="adhoc_pipe_compose_from_template_name",
jinja2_name=template_name,
inputs=inputs or PipeInputSpecFactory.make_empty(),
)
- msg = "Either template_str or template_name must be provided to make_pipe_jinja2_from_template_str"
- raise PipeDefinitionError(msg)
+ else:
+ msg = "Could not make a PipeCompose because neither template_str nor template_name were provided"
+ raise PipeDefinitionError(msg)
diff --git a/pipelex/pipe_operators/pipe_operator.py b/pipelex/pipe_operators/pipe_operator.py
index 75ad892f0..d35a1f411 100644
--- a/pipelex/pipe_operators/pipe_operator.py
+++ b/pipelex/pipe_operators/pipe_operator.py
@@ -38,7 +38,7 @@ class PipeOperator(PipeAbstract, Generic[PipeOperatorOutputType]):
match pipe_run_params.run_mode:
case PipeRunMode.LIVE:
- if self.class_name not in ["PipeJinja2", "PipeLLMPrompt"]:
+ if self.class_name not in ["PipeCompose", "PipeLLMPrompt"]:
name = f"Running [cyan]{self.class_name}[/cyan]"
indent_level = len(pipe_run_params.pipe_stack) - 1
indent = " " * indent_level
diff --git a/tests/integration/pipelex/pipes/test_pipe_jinja2.py b/tests/integration/pipelex/pipes/test_pipe_compose.py
similarity index 61%
rename from tests/integration/pipelex/pipes/test_pipe_jinja2.py
rename to tests/integration/pipelex/pipes/test_pipe_compose.py
index 937ea3cd2..7e34aa303 100644
--- a/tests/integration/pipelex/pipes/test_pipe_jinja2.py
+++ b/tests/integration/pipelex/pipes/test_pipe_compose.py
@@ -8,9 +8,9 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory
from pipelex.core.pipes.pipe_run_params import PipeRunMode
from pipelex.core.pipes.pipe_run_params_factory import PipeRunParamsFactory
from pipelex.hub import get_pipe_router
-from pipelex.pipe_operators.jinja2.pipe_jinja2 import PipeJinja2Output
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
-from pipelex.pipe_operators.jinja2.pipe_jinja2_factory import PipeJinja2Factory
+from pipelex.pipe_operators.compose.pipe_compose import PipeComposeOutput
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
+from pipelex.pipe_operators.compose.pipe_compose_factory import PipeComposeFactory
from pipelex.pipe_works.pipe_job_factory import PipeJobFactory
from pipelex.tools.templating.templating_models import PromptingStyle, TagStyle, TextFormat
from tests.cases import JINJA2TestCases
@@ -18,41 +18,41 @@ from tests.cases import JINJA2TestCases
@pytest.mark.dry_runnable
@pytest.mark.asyncio(loop_scope="class")
-class TestPipeJinja2:
+class TestPipeCompose:
@pytest.mark.parametrize("jinja2", JINJA2TestCases.JINJA2_FOR_ANY)
- async def test_pipe_jinja2_for_any(
+ async def test_pipe_compose_for_any(
self,
pipe_run_mode: PipeRunMode,
jinja2: str,
):
- pipe_jinja2_blueprint = PipeJinja2Blueprint(
+ pipe_compose_blueprint = PipeComposeBlueprint(
definition="Jinja2 test for any context",
jinja2=jinja2,
output=NativeConceptEnum.TEXT,
- extra_context={"place_holder": "[some text from test_pipe_jinja2_for_any]"},
+ extra_context={"place_holder": "[some text from test_pipe_compose_for_any]"},
)
pipe_job = PipeJobFactory.make_pipe_job(
- pipe=PipeJinja2Factory.make_from_blueprint(
+ pipe=PipeComposeFactory.make_from_blueprint(
domain="generic",
- pipe_code="adhoc_for_test_pipe_jinja2_for_any",
- blueprint=pipe_jinja2_blueprint,
+ pipe_code="adhoc_for_test_pipe_compose_for_any",
+ blueprint=pipe_compose_blueprint,
),
pipe_run_params=PipeRunParamsFactory.make_run_params(pipe_run_mode=pipe_run_mode),
)
- pipe_jinja2_output = cast(PipeJinja2Output, await get_pipe_router().run(pipe_job=pipe_job))
- rendered_text = pipe_jinja2_output.rendered_text
+ pipe_compose_output = cast(PipeComposeOutput, await get_pipe_router().run(pipe_job=pipe_job))
+ rendered_text = pipe_compose_output.main_stuff_as_str
pretty_print(rendered_text)
@pytest.mark.parametrize("jinja2", JINJA2TestCases.JINJA2_FOR_STUFF)
- async def test_pipe_jinja2_for_stuff(
+ async def test_pipe_compose_for_stuff(
self,
pipe_run_mode: PipeRunMode,
jinja2: str,
):
- working_memory = WorkingMemoryFactory.make_from_text(text="[some text from test_pipe_jinja2_for_stuff]", name="place_holder")
+ working_memory = WorkingMemoryFactory.make_from_text(text="[some text from test_pipe_compose_for_stuff]", name="place_holder")
- pipe_jinja2_blueprint = PipeJinja2Blueprint(
+ pipe_compose_blueprint = PipeComposeBlueprint(
definition="Jinja2 test for stuff context",
jinja2=jinja2,
prompting_style=PromptingStyle(tag_style=TagStyle.TICKS, text_format=TextFormat.MARKDOWN),
@@ -60,14 +60,14 @@ class TestPipeJinja2:
)
pipe_job = PipeJobFactory.make_pipe_job(
- pipe=PipeJinja2Factory.make_from_blueprint(
+ pipe=PipeComposeFactory.make_from_blueprint(
domain="generic",
- pipe_code="adhoc_for_test_pipe_jinja2",
- blueprint=pipe_jinja2_blueprint,
+ pipe_code="adhoc_for_test_pipe_compose",
+ blueprint=pipe_compose_blueprint,
),
pipe_run_params=PipeRunParamsFactory.make_run_params(pipe_run_mode=pipe_run_mode),
working_memory=working_memory,
)
- pipe_jinja2_output = cast(PipeJinja2Output, await get_pipe_router().run(pipe_job=pipe_job))
- rendered_text = pipe_jinja2_output.rendered_text
+ pipe_compose_output = cast(PipeComposeOutput, await get_pipe_router().run(pipe_job=pipe_job))
+ rendered_text = pipe_compose_output.main_stuff_as_str
pretty_print(rendered_text)
diff --git a/tests/test_pipelines/discord_newsletter.plx b/tests/test_pipelines/discord_newsletter.plx
index eda0b16a9..7c3c3e770 100644
--- a/tests/test_pipelines/discord_newsletter.plx
+++ b/tests/test_pipelines/discord_newsletter.plx
@@ -77,7 +77,7 @@ Keep it short: 200 characters.
"""
[pipe.format_html_newsletter]
-type = "PipeJinja2"
+type = "PipeCompose"
definition = "Combine weekly and channel summaries into a complete newsletter following specific formatting requirements"
inputs = { weekly_summary = "Text", channel_summaries = "ChannelSummary" }
output = "HtmlNewsletter"
diff --git a/tests/test_pipelines/misc_tests/test_jinja2.plx b/tests/test_pipelines/misc_tests/test_jinja2.plx
index 9a0d44398..c1e362956 100644
--- a/tests/test_pipelines/misc_tests/test_jinja2.plx
+++ b/tests/test_pipelines/misc_tests/test_jinja2.plx
@@ -2,7 +2,7 @@ domain = "test_jinja2"
definition = "This library is intended for testing Jinja2"
[pipe.jinja2_test_1]
-type = "PipeJinja2"
+type = "PipeCompose"
definition = "Jinja2 test 1"
inputs = { text = "Text" }
output = "Text"
diff --git a/tests/test_pipelines/tricky_questions.plx b/tests/test_pipelines/tricky_questions.plx
index 798495bc8..654702273 100644
--- a/tests/test_pipelines/tricky_questions.plx
+++ b/tests/test_pipelines/tricky_questions.plx
@@ -47,7 +47,7 @@ Answer in 4 parts:
"""
[pipe.conclude_thoughtful_answer]
-type = "PipeJinja2"
+type = "PipeCompose"
definition = "Conclude a thoughtful answer"
inputs = { thoughtful_answer = "ThoughtfulAnswer" }
output = "ThoughtfulAnswerConclusion"
diff --git a/tests/unit/pipelex/core/test_data/__init__.py b/tests/unit/pipelex/core/test_data/__init__.py
index 1df3bcca0..ac9101a9f 100644
--- a/tests/unit/pipelex/core/test_data/__init__.py
+++ b/tests/unit/pipelex/core/test_data/__init__.py
@@ -14,7 +14,7 @@ from .pipes.controllers.parallel.pipe_parallel import PIPE_PARALLEL_TEST_CASES
from .pipes.controllers.sequence.pipe_sequence import PIPE_SEQUENCE_TEST_CASES
from .pipes.operators.func.pipe_func import PIPE_FUNC_TEST_CASES
from .pipes.operators.img_gen.pipe_img_gen import PIPE_IMG_GEN_TEST_CASES
-from .pipes.operators.jinja2.pipe_jinja2 import PIPE_JINJA2_TEST_CASES
+from .pipes.operators.compose.pipe_compose import PIPE_COMPOSE_TEST_CASES
from .pipes.operators.llm.pipe_llm import PIPE_LLM_TEST_CASES
from .pipes.operators.ocr.pipe_ocr import PIPE_OCR_TEST_CASES
@@ -35,7 +35,7 @@ class InterpreterTestCases:
*PIPE_OCR_TEST_CASES,
*PIPE_FUNC_TEST_CASES,
*PIPE_IMG_GEN_TEST_CASES,
- *PIPE_JINJA2_TEST_CASES,
+ *PIPE_COMPOSE_TEST_CASES,
# Pipe controller tests
*PIPE_SEQUENCE_TEST_CASES,
*PIPE_CONDITION_TEST_CASES,
diff --git a/tests/unit/pipelex/core/test_data/pipes/operators/jinja2/__init__.py b/tests/unit/pipelex/core/test_data/pipes/operators/compose/__init__.py
similarity index 100%
rename from tests/unit/pipelex/core/test_data/pipes/operators/jinja2/__init__.py
rename to tests/unit/pipelex/core/test_data/pipes/operators/compose/__init__.py
diff --git a/tests/unit/pipelex/core/test_data/pipes/operators/jinja2/pipe_jinja2.py b/tests/unit/pipelex/core/test_data/pipes/operators/compose/pipe_compose.py
similarity index 72%
rename from tests/unit/pipelex/core/test_data/pipes/operators/jinja2/pipe_jinja2.py
rename to tests/unit/pipelex/core/test_data/pipes/operators/compose/pipe_compose.py
index b020bdcd3..a097edae4 100644
--- a/tests/unit/pipelex/core/test_data/pipes/operators/jinja2/pipe_jinja2.py
+++ b/tests/unit/pipelex/core/test_data/pipes/operators/compose/pipe_compose.py
@@ -1,15 +1,15 @@
from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint
from pipelex.core.concepts.concept_native import NativeConceptEnum
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
from pipelex.tools.templating.jinja2_template_category import Jinja2TemplateCategory
-PIPE_JINJA2 = (
- "pipe_jinja2",
+PIPE_COMPOSE = (
+ "pipe_compose",
"""domain = "test_pipes"
definition = "Domain with template processing pipe"
[pipe.process_template]
-type = "PipeJinja2"
+type = "PipeCompose"
definition = "Process a Jinja2 template"
output = "Text"
jinja2 = "Hello {{ name }}!"
@@ -19,8 +19,8 @@ template_category = "markdown"
domain="test_pipes",
definition="Domain with template processing pipe",
pipe={
- "process_template": PipeJinja2Blueprint(
- type="PipeJinja2",
+ "process_template": PipeComposeBlueprint(
+ type="PipeCompose",
definition="Process a Jinja2 template",
output=NativeConceptEnum.TEXT,
jinja2="Hello {{ name }}!",
@@ -30,7 +30,7 @@ template_category = "markdown"
),
)
-# Export all PipeJinja2 test cases
-PIPE_JINJA2_TEST_CASES = [
- PIPE_JINJA2,
+# Export all PipeCompose test cases
+PIPE_COMPOSE_TEST_CASES = [
+ PIPE_COMPOSE,
]
diff --git a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_data.py b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_data.py
index 32a2a4d96..28616ce78 100644
--- a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_data.py
+++ b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_data.py
@@ -2,27 +2,27 @@ from typing import ClassVar
from pipelex.core.pipes.pipe_input_blueprint import InputRequirementBlueprint
from pipelex.libraries.pipelines.builder.pipe.inputs_spec import InputRequirementSpec
-from pipelex.libraries.pipelines.builder.pipe.pipe_jinja2_spec import PipeJinja2Spec, PromptingStyleSpec
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
+from pipelex.libraries.pipelines.builder.pipe.pipe_compose_spec import PipeComposeSpec, PromptingStyleSpec
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
from pipelex.tools.templating.jinja2_template_category import Jinja2TemplateCategory
from pipelex.tools.templating.templating_models import PromptingStyle, TagStyle, TextFormat
-class PipeJinja2TestCases:
+class PipeComposeTestCases:
SIMPLE_JINJA2 = (
"simple_jinja2",
- PipeJinja2Spec(
+ PipeComposeSpec(
the_pipe_code="template_renderer",
definition="Render a template",
inputs={"data": InputRequirementSpec(concept="Data")},
output="RenderedText",
jinja2="Hello {{ data.name }}!",
),
- PipeJinja2Blueprint(
+ PipeComposeBlueprint(
definition="Render a template",
inputs={"data": InputRequirementBlueprint(concept="Data")},
output="RenderedText",
- type="PipeJinja2",
+ type="PipeCompose",
category="PipeOperator",
jinja2_name=None,
jinja2="Hello {{ data.name }}!",
@@ -34,7 +34,7 @@ class PipeJinja2TestCases:
JINJA2_WITH_STYLE = (
"jinja2_with_style",
- PipeJinja2Spec(
+ PipeComposeSpec(
the_pipe_code="styled_template",
definition="Template with prompting style",
inputs={"input": InputRequirementSpec(concept="Input")},
@@ -47,11 +47,11 @@ class PipeJinja2TestCases:
template_category=Jinja2TemplateCategory.MARKDOWN,
extra_context={"version": "1.0"},
),
- PipeJinja2Blueprint(
+ PipeComposeBlueprint(
definition="Template with prompting style",
inputs={"input": InputRequirementBlueprint(concept="Input")},
output="Output",
- type="PipeJinja2",
+ type="PipeCompose",
category="PipeOperator",
jinja2_name="custom_template",
jinja2=None,
@@ -64,7 +64,7 @@ class PipeJinja2TestCases:
),
)
- TEST_CASES: ClassVar[list[tuple[str, PipeJinja2Spec, PipeJinja2Blueprint]]] = [
+ TEST_CASES: ClassVar[list[tuple[str, PipeComposeSpec, PipeComposeBlueprint]]] = [
SIMPLE_JINJA2,
JINJA2_WITH_STYLE,
]
diff --git a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_pipe_jinja2.py b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_pipe_jinja2.py
index a0afd4e26..cf72ba612 100644
--- a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_pipe_jinja2.py
+++ b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_jinja2/test_pipe_jinja2.py
@@ -1,21 +1,21 @@
import pytest
-from pipelex.libraries.pipelines.builder.pipe.pipe_jinja2_spec import PipeJinja2Spec
-from pipelex.pipe_operators.jinja2.pipe_jinja2_blueprint import PipeJinja2Blueprint
+from pipelex.libraries.pipelines.builder.pipe.pipe_compose_spec import PipeComposeSpec
+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint
-from .test_data import PipeJinja2TestCases
+from .test_data import PipeComposeTestCases
-class TestPipeJinja2BlueprintConversion:
+class TestPipeComposeBlueprintConversion:
@pytest.mark.parametrize(
"test_name,pipe_spec,expected_blueprint",
- PipeJinja2TestCases.TEST_CASES,
+ PipeComposeTestCases.TEST_CASES,
)
- def test_pipe_jinja2_spec_to_blueprint(
+ def test_pipe_compose_spec_to_blueprint(
self,
test_name: str,
- pipe_spec: PipeJinja2Spec,
- expected_blueprint: PipeJinja2Blueprint,
+ pipe_spec: PipeComposeSpec,
+ expected_blueprint: PipeComposeBlueprint,
):
result = pipe_spec.to_blueprint()
assert result == expected_blueprint