Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Refactor/epic 2 pipe ocr input and agent rules (#300)

Louis Choquel969c29a3c3bc
diff --git a/.cursor/rules/coding_standards.mdc b/.cursor/rules/coding_standards.mdc
index 6c6d06f8f..7e504756c 100644
--- a/.cursor/rules/coding_standards.mdc
+++ b/.cursor/rules/coding_standards.mdc
@@ -10,10 +10,16 @@ This document outlines the core coding standards, best practices, and quality co
## Type Hints
1. **Always Use Type Hints**
- - Every function parameter must be typed
- - Every function return must be typed
- - Use type hints for all variables where type is not obvious
- - Use types with Uppercase first letter (Dict[], List[], etc.)
+
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals
+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`
+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
2. **StrEnum**
- Import from `pipelex.types`:
@@ -21,19 +27,34 @@ This document outlines the core coding standards, best practices, and quality co
from pipelex.types import StrEnum
```
-## BaseModel Standards
-
-- Respect Pydantic v2 standards
-- Keep models focused and single-purpose
-- Use descriptive field names
-- Use type hints for all fields
-- Document complex validations
-- Use Optional[] for nullable fields
-- Use Field(default_factory=...) for mutable defaults
+3. **Self type**
+ - Import from `pipelex.types`:
+ ```python
+ from pipelex.types import Self
+ ```
## Factory Pattern
-- Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
+
+## Error Handling
+
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones
+ - Never catch Exception, only catch specific exceptions
+ - Always add `from exc` to the exception
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
+
+ **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
## Documentation
@@ -61,21 +82,6 @@ This document outlines the core coding standards, best practices, and quality co
"""
```
-## Error Handling
-
-1. **Graceful Error Handling**
- - Use try/except blocks with specific exceptions
- - Convert third-party exceptions to custom ones
- ```python
- try:
- from fal_client import AsyncClient as FalAsyncClient
- except ImportError as exc:
- raise MissingDependencyError(
- "fal-client", "fal",
- "The fal-client SDK is required to use FAL models."
- ) from exc
- ```
-
## Code Quality Checks
### Linting and Type Checking
diff --git a/.cursor/rules/docs.mdc b/.cursor/rules/docs.mdc
index 31461aa94..ac9c5191e 100644
--- a/.cursor/rules/docs.mdc
+++ b/.cursor/rules/docs.mdc
@@ -6,3 +6,7 @@ alwaysApply: false
Write docs and answer questions about writing docs.
We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
+
+## MkDocs Markdown Requirements
+
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
diff --git a/.cursor/rules/llms.mdc b/.cursor/rules/llms.mdc
index 7f1d75f7b..54de9adcf 100644
--- a/.cursor/rules/llms.mdc
+++ b/.cursor/rules/llms.mdc
@@ -21,10 +21,10 @@ An llm_handle can be either:
```toml
[aliases]
-best-claude = "claude-4.1-opus"
-best-gemini = "gemini-2.5-pro"
-best-mistral = "mistral-large"
+base-claude = "claude-4-sonnet"
base-gpt = "gpt-5"
+base-gemini = "gemini-2.5-flash"
+base-mistral = "mistral-medium"
```
The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
@@ -38,7 +38,7 @@ Here is an example of using an llm_handle to specify which LLM to use in a PipeL
type = "PipeLLM"
definition = "Write text about Hello World."
output = "Text"
-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }
+llm = { llm_handle = "gpt-5", temperature = 0.9 }
prompt_template = """
Write a haiku about Hello World.
"""
@@ -52,7 +52,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp
Examples:
```toml
-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }
+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }
llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
```
@@ -78,4 +78,4 @@ The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_
You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.
+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
diff --git a/.cursor/rules/pipelex.mdc b/.cursor/rules/pipelex.mdc
index 3020740a7..52e8a88de 100644
--- a/.cursor/rules/pipelex.mdc
+++ b/.cursor/rules/pipelex.mdc
@@ -66,11 +66,11 @@ definition = "....."
```
The pipes will all have at least this base structure.
-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).
+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['ocr_input']'``
-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.
+variable='['invoice']'``
+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
<NEVER DO THIS>
@@ -164,7 +164,8 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ
2. **Operators** - For specific tasks:
- `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeOcr` - OCR Pipe
+ - `PipeOcr` - Extract text and images from an image or a PDF
+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- `PipeImgGen` - Generate Images
- `PipeFunc` - For running classic python scripts
@@ -354,20 +355,20 @@ prompt_template = "Describe what you see in this image"
## Purpose
-Extract text and images from an image or a PDF
+The PipeOcr operator is used to extract text and images from an image or a PDF
## Basic Usage
-### Simple Text Generation
+### Simple Text Extraction
```plx
[pipe.extract_info]
type = "PipeOcr"
definition = "extract the information"
-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input
+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
output = "Page"
```
-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.
+Only one input is allowed and it must either be an `Image` or a `PDF`.
The output concept `Page` is a native concept, with the structure `PageContent`:
It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`
@@ -384,6 +385,264 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"
- `text_and_images` are the text, and the related images found in the input image or PDF.
- `page_view` is the screenshot of the whole pdf page/image.
+# PipeCompose Guide
+
+## Purpose
+
+The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
+
+## Basic Usage
+
+### Simple Template Composition
+```plx
+[pipe.compose_report]
+type = "PipeCompose"
+definition = "Compose a report using template"
+inputs = { data = "ReportData" }
+output = "Text"
+jinja2 = """
+# Report Summary
+
+Based on the analysis:
+@data
+
+Generated on: {{ current_date }}
+"""
+```
+
+### Using Named Templates
+```plx
+[pipe.use_template]
+type = "PipeCompose"
+definition = "Use a predefined template"
+inputs = { content = "Text" }
+output = "Text"
+jinja2_name = "standard_report_template"
+```
+
+### CRM Email Template
+```plx
+[pipe.compose_follow_up_email]
+type = "PipeCompose"
+definition = "Compose a personalized follow-up email for CRM"
+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
+output = "Text"
+template_category = "html"
+prompting_style = { tag_style = "square_brackets", text_format = "html" }
+jinja2 = """
+Subject: Following up on our $deal.product_name discussion
+
+Hi $customer.first_name,
+
+I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+
+Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+
+I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+
+**Next Steps:**
+- Schedule a demo tailored to your specific needs
+- Provide you with a customized quote based on your requirements
+- Connect you with our implementation team
+
+Would you be available for a 30-minute call this week? I have openings on:
+{% for slot in available_slots %}
+- {{ slot }}
+{% endfor %}
+
+Looking forward to moving this forward together!
+
+Best regards,
+$sales_rep.name
+$sales_rep.title
+$sales_rep.phone | $sales_rep.email
+"""
+```
+
+## Key Parameters
+
+- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)
+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)
+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
+- `prompting_style`: Styling options for template rendering
+- `extra_context`: Additional context variables for template
+
+## Template Variables
+
+Use the same variable insertion rules as PipeLLM:
+- `@variable` for block insertion (multi-line content)
+- `$variable` for inline insertion (short text)
+
+# PipeImgGen Guide
+
+## Purpose
+
+The PipeImgGen operator is used to generate images using AI image generation models.
+
+## Basic Usage
+
+### Simple Image Generation
+```plx
+[pipe.generate_image]
+type = "PipeImgGen"
+definition = "Generate an image from prompt"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+```
+
+### Using Image Generation Settings
+```plx
+[pipe.generate_photo]
+type = "PipeImgGen"
+definition = "Generate a high-quality photo"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Photo"
+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }
+aspect_ratio = "16:9"
+nb_steps = 8
+```
+
+### Multiple Image Generation
+```plx
+[pipe.generate_variations]
+type = "PipeImgGen"
+definition = "Generate multiple image variations"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+nb_output = 3
+seed = "auto"
+```
+
+### Advanced Configuration
+```plx
+[pipe.generate_custom]
+type = "PipeImgGen"
+definition = "Generate image with custom settings"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+img_gen = "img_gen_preset_name" # Use predefined preset
+aspect_ratio = "1:1"
+quality = "hd"
+background = "transparent"
+output_format = "png"
+is_raw = false
+safety_tolerance = 3
+```
+
+## Key Parameters
+
+### Image Generation Settings
+- `img_gen`: ImgGenChoice (preset name or inline settings)
+- `img_gen_handle`: Direct model handle (legacy)
+- `quality`: Image quality ("standard", "hd")
+- `nb_steps`: Number of generation steps
+- `guidance_scale`: How closely to follow the prompt
+
+### Output Configuration
+- `nb_output`: Number of images to generate
+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
+- `output_format`: File format ("png", "jpeg", "webp")
+- `background`: Background type ("default", "transparent")
+
+### Generation Control
+- `seed`: Random seed (integer or "auto")
+- `is_raw`: Whether to apply post-processing
+- `is_moderated`: Enable content moderation
+- `safety_tolerance`: Content safety level (1-6)
+
+## Input Requirements
+
+PipeImgGen requires exactly one input that must be either:
+- An `ImgGenPrompt` concept
+- A concept that refines `ImgGenPrompt`
+
+The input can be named anything but must contain the prompt text for image generation.
+
+# PipeFunc Guide
+
+## Purpose
+
+The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+
+## Basic Usage
+
+### Simple Function Call
+```plx
+[pipe.process_data]
+type = "PipeFunc"
+definition = "Process data using custom function"
+inputs = { input_data = "DataType" }
+output = "ProcessedData"
+function_name = "process_data_function"
+```
+
+### File Processing Example
+```plx
+[pipe.read_file]
+type = "PipeFunc"
+definition = "Read file content"
+inputs = { file_path = "FilePath" }
+output = "FileContent"
+function_name = "read_file_content"
+```
+
+## Key Parameters
+
+- `function_name`: Name of the Python function to call (must be registered in func_registry)
+
+## Function Requirements
+
+The Python function must:
+
+1. **Be registered** in the `func_registry`
+2. **Accept `working_memory`** as a parameter:
+ ```python
+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
+ # Function implementation
+ pass
+ ```
+
+3. **Return appropriate types**:
+ - `StuffContent`: Single content object
+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)
+ - `str`: Simple string (becomes TextContent)
+
+## Function Registration
+
+Functions must be registered in the function registry before use:
+
+```python
+from pipelex.tools.func_registry import func_registry
+
+@func_registry.register("my_function_name")
+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
+ # Access inputs from working memory
+ input_data = working_memory.get_stuff("input_name")
+
+ # Process data
+ result = process_logic(input_data.content)
+
+ # Return result
+ return MyResultContent(data=result)
+```
+
+## Working Memory Access
+
+Inside the function, access pipeline inputs through working memory:
+
+```python
+async def process_function(working_memory: WorkingMemory) -> TextContent:
+ # Get input stuff by name
+ input_stuff = working_memory.get_stuff("input_name")
+
+ # Access the content
+ input_content = input_stuff.content
+
+ # Process and return
+ processed_text = f"Processed: {input_content.text}"
+ return TextContent(text=processed_text)
+```
+
This rule explains how to write prompt templates in PipeLLM definitions.
## Insert stuff inside a tagged block
@@ -427,17 +686,47 @@ Be sure to make the proper choice of prefix for each insertion.
**DO NOT write "$topic" alone in an isolated line.**
**DO write things like "Write an essay about $topic" included in an actual sentence.**
-# Example to execute a pipeline
+# Guide to write an example to execute a pipeline
+
+## Example to execute a pipeline with text output
```python
import asyncio
from pipelex import pretty_print
-from pipelex.hub import get_pipeline_tracker, get_report_delegate
from pipelex.pipelex import Pipelex
from pipelex.pipeline.execute import execute_pipeline
-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart
+
+async def hello_world() -> str:
+ """
+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.
+ """
+ # Run the pipe
+ pipe_output = await execute_pipeline(
+ pipe_code="hello_world",
+ )
+
+ return pipe_output.main_stuff_as_str
+
+
+# start Pipelex
+Pipelex.make()
+# run sample using asyncio
+output_text = asyncio.run(hello_world())
+pretty_print(output_text, title="Your first Pipelex output")
+```
+
+## Example to execute a pipeline with structured output
+
+```python
+import asyncio
+
+from pipelex import pretty_print
+from pipelex.pipelex import Pipelex
+from pipelex.pipeline.execute import execute_pipeline
+
+from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChart
SAMPLE_NAME = "extract_gantt"
IMAGE_URL = "assets/gantt/gantt_tree_house.png"
@@ -462,15 +751,14 @@ async def extract_gantt(image_url: str) -> GanttChart:
Pipelex.make()
# run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))
-
-# Display cost report (tokens used and cost)
-get_report_delegate().generate_report()
-# output results
+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
pretty_print(gantt_chart, title="Gantt Chart")
-get_pipeline_tracker().output_flowchart()
```
+## Setting up the input memory
+
+### Explanation of input memory
+
The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
```python
StuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | str
@@ -496,7 +784,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
pipe_output = await execute_pipeline(
pipe_code="power_extractor_dpe",
input_memory={
- "ocr_input": PDFContent(url=pdf_url),
+ "document": PDFContent(url=pdf_url),
},
)
@@ -537,6 +825,168 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
)
```
+## Using the outputs of a pipeline
+
+All pipe executions return a `PipeOutput` object.
+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
+
+```python
+
+class PipeOutput(BaseModel):
+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
+
+ @property
+ def main_stuff(self) -> Stuff:
+ ...
+
+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
+ ...
+
+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
+ ...
+
+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
+ ...
+
+ @property
+ def main_stuff_as_text(self) -> TextContent:
+ ...
+
+ @property
+ def main_stuff_as_str(self) -> str:
+ ...
+
+ @property
+ def main_stuff_as_image(self) -> ImageContent:
+ ...
+
+ @property
+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
+ ...
+
+ @property
+ def main_stuff_as_number(self) -> NumberContent:
+ ...
+
+ @property
+ def main_stuff_as_html(self) -> HtmlContent:
+ ...
+
+ @property
+ def main_stuff_as_mermaid(self) -> MermaidContent:
+ ...
+```
+
+As you can see, you can extarct any variable from the output working memory.
+
+### Getting the main stuff as a specific type
+
+Simple text as a string:
+
+```python
+result = pipe_output.main_stuff_as_str
+```
+Structured object (BaseModel):
+
+```python
+result = pipe_output.main_stuff_as(content_type=GanttChart)
+```
+
+If it's a list, you can get a `ListContent` of the specific type.
+
+```python
+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
+```
+
+or if you want, you can get the actual items as a regular python list:
+
+```python
+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
+```
+
+---
+
+## Rules to choose LLM models used in PipeLLMs.
+
+### LLM Configuration System
+
+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+
+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
+- **Routing**: `.pipelex/inference/routing_profiles.toml`
+
+### LLM Handles
+
+An llm_handle can be either:
+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+
+```toml
+[aliases]
+base-claude = "claude-4-sonnet"
+base-gpt = "gpt-5"
+base-gemini = "gemini-2.5-flash"
+base-mistral = "mistral-medium"
+```
+
+The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+
+### Using an LLM Handle in a PipeLLM
+
+Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
+
+```plx
+[pipe.hello_world]
+type = "PipeLLM"
+definition = "Write text about Hello World."
+output = "Text"
+llm = { llm_handle = "gpt-5", temperature = 0.9 }
+prompt_template = """
+Write a haiku about Hello World.
+"""
+```
+
+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
+
+### LLM Presets
+
+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+
+Examples:
+```toml
+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }
+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```
+
+The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+
+```plx
+[pipe.extract_invoice]
+type = "PipeLLM"
+definition = "Extract invoice information from an invoice text transcript"
+inputs = { invoice_text = "InvoiceText" }
+output = "Invoice"
+llm = "llm_to_extract_invoice"
+prompt_template = """
+Extract invoice information from this invoice:
+
+The category of this invoice is: $invoice_details.category.
+
+@invoice_text
+"""
+```
+
+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+
+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+
+---
+
ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
Then, create an example file to run the pipeline in the `examples` folder.
But don't write documentation unless asked explicitly to.
diff --git a/.cursor/rules/pytest.mdc b/.cursor/rules/pytest.mdc
index ca31b9632..53db2a504 100644
--- a/.cursor/rules/pytest.mdc
+++ b/.cursor/rules/pytest.mdc
@@ -3,6 +3,12 @@ description:
globs: tests/**/*.py
alwaysApply: false
---
+# Writing unit tests
+
+## Unit test generalities
+
+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
+
These rules apply when writing unit tests.
- Always use pytest
@@ -30,9 +36,9 @@ Apply the appropriate markers:
Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-## Tips
+### Important rules
-- Never use the unittest.mock. Use pytest-mock
+- Never use the unittest.mock. Use pytest-mock.
## Test Class Structure
@@ -61,9 +67,9 @@ class TestFooBar:
Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-# Pipe tests
+## Writing integration test to test pipes
-## Required imports for pipe tests
+### Required imports for pipe tests
```python
import pytest
@@ -82,7 +88,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParams
from pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol
```
-## Pipe test implementation steps
+### Pipe test implementation steps
1. Create Stuff from blueprint:
@@ -112,14 +118,7 @@ pipe_output = await pipe_router.run_pipe(
)
```
-4. Log output and generate report:
-
-```python
-pretty_print(pipe_output, title=f"Pipe output")
-get_report_delegate().generate_report()
-```
-
-5. Basic assertions:
+4. Basic assertions:
```python
assert pipe_output is not None
@@ -127,7 +126,7 @@ assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
```
-## Test Data Organization
+### Test Data Organization
- If it's not already there, create a `test_data.py` file in the test directory
- Define test cases using `StuffBlueprint`:
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index b911fe4fd..57111a045 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -1,4 +1,3 @@
-Concatenation
# Coding Standards & Best Practices
This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co
## Type Hints
1. **Always Use Type Hints**
- - Every function parameter must be typed
- - Every function return must be typed
- - Use type hints for all variables where type is not obvious
- - Use types with Uppercase first letter (Dict[], List[], etc.)
+
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals
+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`
+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
2. **StrEnum**
- Import from `pipelex.types`:
@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality co
from pipelex.types import StrEnum
```
-## BaseModel Standards
-
-- Respect Pydantic v2 standards
-- Keep models focused and single-purpose
-- Use descriptive field names
-- Use type hints for all fields
-- Document complex validations
-- Use Optional[] for nullable fields
-- Use Field(default_factory=...) for mutable defaults
+3. **Self type**
+ - Import from `pipelex.types`:
+ ```python
+ from pipelex.types import Self
+ ```
## Factory Pattern
-- Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
+
+## Error Handling
+
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones
+ - Never catch Exception, only catch specific exceptions
+ - Always add `from exc` to the exception
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
+
+ **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
## Documentation
@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co
"""
```
-## Error Handling
-
-1. **Graceful Error Handling**
- - Use try/except blocks with specific exceptions
- - Convert third-party exceptions to custom ones
- ```python
- try:
- from fal_client import AsyncClient as FalAsyncClient
- except ImportError as exc:
- raise MissingDependencyError(
- "fal-client", "fal",
- "The fal-client SDK is required to use FAL models."
- ) from exc
- ```
-
## Code Quality Checks
### Linting and Type Checking
@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.
- **Pipelines**: `pipelex/libraries/pipelines/`
- **Tests**: `tests/` directory
- **Documentation**: `docs/` directory
-
# Pipeline Guide
+
+---
+
+# Guide to write or edit pipelines using the Pipelex language in .plx files
- Always first write your "plan" in natural langage, then transcribe it in pipelex.
-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)
-# Pipeline Structure Guide
-
## Pipeline File Naming
- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
- Files must be `.py` for structures
@@ -183,11 +189,11 @@ definition = "....."
```
The pipes will all have at least this base structure.
-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).
+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['ocr_input']'``
-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.
+variable='['invoice']'``
+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
<NEVER DO THIS>
@@ -201,9 +207,9 @@ inputs = {
- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-# Structured Models Rules
+## Structuring Models
-## Model Location and Registration
+### Model Location and Registration
- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`
- Models must inherit from `StructuredContent` or appropriate content type
@@ -232,7 +238,6 @@ If you simply need to refine another native concept, construct it like this:
[concept.Landscape]
refines = "Image"
```
-
Only create a Python structure class when you need to add specific fields:
```python
@@ -255,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent
# Date fields should remove timezone
date_field: Optional[datetime] = None
```
-## Usage
+### Usage
Structures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.
Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.
-## Best Practices for structures
+### Best Practices for structures
- Respect Pydantic v2 standards
- Use type hints for all fields
- Use `Field` declaration and write the description
-## Pipe Controllers and Pipe Operator
+## Pipe Controllers and Pipe Operators
Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
@@ -277,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ
- `PipeSequence` - For creating a sequence of multiple steps
- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- `PipeParallel` - For parallelizing pipes
- - `PipeBatch` - For running pipes in Batch over a ListContent
2. **Operators** - For specific tasks:
- `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeOcr` - OCR Pipe
+ - `PipeOcr` - Extract text and images from an image or a PDF
+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- `PipeImgGen` - Generate Images
- `PipeFunc` - For running classic python scripts
-# PipeSequence Guide
+## PipeSequence controller
-## Purpose
-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.
+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-## Basic Structure
+### Basic Structure
```plx
[pipe.your_sequence_name]
type = "PipeSequence"
@@ -304,13 +308,13 @@ steps = [
]
```
-## Key Components
+### Key Components
1. **Steps Array**: List of pipes to execute in sequence
- `pipe`: Name of the pipe to execute
- `result`: Name to assign to the pipe's output that will be in the working memory
-## Using PipeBatch in Steps
+### Using PipeBatch in Steps
You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
@@ -331,13 +335,11 @@ steps = [
The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-# PipeCondition Controller
+## PipeCondition controller
The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-## Usage in PLX Configuration
-
-### Basic Usage with Direct Expression
+### Basic usage
```plx
[pipe.conditional_operation]
@@ -367,7 +369,7 @@ medium = "process_medium"
large = "process_large"
```
-## Key Parameters
+### Key Parameters
- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
@@ -375,41 +377,15 @@ large = "process_large"
1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.
2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger
-# PipeBatch Controller
-
-The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.
-
-## Usage in PLX Configuration
-
-```plx
-[pipe.sequence_with_batch]
-type = "PipeSequence"
-definition = "A Sequence of pipes"
-inputs = { input_data = "ConceptName" }
-output = "OutputConceptName"
-steps = [
- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }
-]
-```
-
-## Key Parameters
-
-- `pipe`: The pipe operation to apply to each element in the batch
-- `batch_over`: The name of the list in the context to iterate over
-- `batch_as`: The name to use for the current element in the pipe's context
-- `result`: Where to store the results of the batch operation
-
-# PipeLLM Guide
-
-## Purpose
+## PipeLLM operator
PipeLLM is used to:
1. Generate text or objects with LLMs
2. Process images with Vision LLMs
-## Basic Usage
+### Basic Usage
-### Simple Text Generation
+Simple Text Generation:
```plx
[pipe.write_story]
type = "PipeLLM"
@@ -420,7 +396,7 @@ Write a short story about a programmer.
"""
```
-### Structured Data Extraction
+Structured Data Extraction:
```plx
[pipe.extract_info]
type = "PipeLLM"
@@ -433,8 +409,7 @@ Extract person information from this text:
"""
```
-### System Prompts
-Add system-level instructions:
+Supports system instructions:
```plx
[pipe.expert_analysis]
type = "PipeLLM"
@@ -445,46 +420,95 @@ prompt_template = "Analyze this data"
```
### Multiple Outputs
-Generate multiple results:
+
+Generate multiple outputs (fixed number):
```plx
[pipe.generate_ideas]
type = "PipeLLM"
definition = "Generate ideas"
output = "Idea"
nb_output = 3 # Generate exactly 3 ideas
-# OR
+```
+
+Generate multiple outputs (variable number):
+```plx
+[pipe.generate_ideas]
+type = "PipeLLM"
+definition = "Generate ideas"
+output = "Idea"
multiple_output = true # Let the LLM decide how many to generate
```
-### Vision Tasks
+### Vision
+
Process images with VLMs:
```plx
[pipe.analyze_image]
type = "PipeLLM"
definition = "Analyze image"
-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }
+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }
output = "ImageAnalysis"
prompt_template = "Describe what you see in this image"
```
-# PipeOCR Guide
+### Writing prompts for PipeLLM
+
+**Insert stuff inside a tagged block**
+
+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+
+Example template:
+```plx
+prompt_template = """
+Match the expense with its corresponding invoice:
+
+@expense
+
+@invoices
+"""
+```
+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
+
+DO NOT write things like "Here is the expense: @expense".
+DO write simply "@expense" alone in an isolated line.
-## Purpose
+**Insert stuff inline**
-Extract text and images from an image or a PDF
+If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+
+Example template:
+```plx
+prompt_template = """
+Your goal is to summarize everything related to $topic in the provided text:
+
+@text
+
+Please provide only the summary, with no additional text or explanations.
+Your summary should not be longer than 2 sentences.
+"""
+```
+
+In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
+Be sure to make the proper choice of prefix for each insertion.
-## Basic Usage
+DO NOT write "$topic" alone in an isolated line.
+DO write things like "Write an essay about $topic" to include text into an actual sentence.
-### Simple Text Generation
+
+## PipeOcr operator
+
+The PipeOcr operator is used to extract text and images from an image or a PDF
+
+### Simple Text Extraction
```plx
[pipe.extract_info]
type = "PipeOcr"
definition = "extract the information"
-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input
+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
output = "Page"
```
-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.
+Only one input is allowed and it must either be an `Image` or a `PDF`.
The output concept `Page` is a native concept, with the structure `PageContent`:
It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`
@@ -501,60 +525,386 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"
- `text_and_images` are the text, and the related images found in the input image or PDF.
- `page_view` is the screenshot of the whole pdf page/image.
-This rule explains how to write prompt templates in PipeLLM definitions.
+## PipeCompose operator
-## Insert stuff inside a tagged block
+The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+### Basic Usage
-Example template:
+Simple Template Composition:
```plx
-prompt_template = """
-Match the expense with its corresponding invoice:
+[pipe.compose_report]
+type = "PipeCompose"
+definition = "Compose a report using template"
+inputs = { data = "ReportData" }
+output = "Text"
+jinja2 = """
+# Report Summary
-@expense
+Based on the analysis:
+$data
-@invoices
+Generated on: {{ current_date }}
"""
```
-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
-**DO NOT write things like "Here is the expense: @expense".**
-**DO write simply "@expense" alone in an isolated line.**
+Using Named Templates:
+```plx
+[pipe.use_template]
+type = "PipeCompose"
+definition = "Use a predefined template"
+inputs = { content = "Text" }
+output = "Text"
+jinja2_name = "standard_report_template"
+```
-## Insert stuff inline
+CRM Email Template:
+```plx
+[pipe.compose_follow_up_email]
+type = "PipeCompose"
+definition = "Compose a personalized follow-up email for CRM"
+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
+output = "Text"
+template_category = "html"
+prompting_style = { tag_style = "square_brackets", text_format = "html" }
+jinja2 = """
+Subject: Following up on our $deal.product_name discussion
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+Hi $customer.first_name,
+
+I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+
+Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+
+I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+
+**Next Steps:**
+- Schedule a demo tailored to your specific needs
+- Provide you with a customized quote based on your requirements
+- Connect you with our implementation team
+
+Would you be available for a 30-minute call this week? I have openings on:
+{% for slot in available_slots %}
+- {{ slot }}
+{% endfor %}
+
+Looking forward to moving this forward together!
+
+Best regards,
+$sales_rep.name
+$sales_rep.title
+$sales_rep.phone | $sales_rep.email
+"""
+```
+
+### Key Parameters
+
+- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)
+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)
+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
+- `prompting_style`: Styling options for template rendering
+- `extra_context`: Additional context variables for template
+
+### Template Variables
+
+Use the same variable insertion rules as PipeLLM:
+- `@variable` for block insertion (multi-line content)
+- `$variable` for inline insertion (short text)
+
+## PipeImgGen operator
+
+The PipeImgGen operator is used to generate images using AI image generation models.
+
+### Basic Usage
+
+Simple Image Generation:
+```plx
+[pipe.generate_image]
+type = "PipeImgGen"
+definition = "Generate an image from prompt"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+```
+
+Using Image Generation Settings:
+```plx
+[pipe.generate_photo]
+type = "PipeImgGen"
+definition = "Generate a high-quality photo"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Photo"
+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }
+aspect_ratio = "16:9"
+nb_steps = 8
+```
+
+Multiple Image Generation:
+```plx
+[pipe.generate_variations]
+type = "PipeImgGen"
+definition = "Generate multiple image variations"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+nb_output = 3
+seed = "auto"
+```
+
+Advanced Configuration:
+```plx
+[pipe.generate_custom]
+type = "PipeImgGen"
+definition = "Generate image with custom settings"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+img_gen = "img_gen_preset_name" # Use predefined preset
+aspect_ratio = "1:1"
+quality = "hd"
+background = "transparent"
+output_format = "png"
+is_raw = false
+safety_tolerance = 3
+```
+
+### Key Parameters
+
+**Image Generation Settings:**
+- `img_gen`: ImgGenChoice (preset name or inline settings)
+- `img_gen_handle`: Direct model handle (legacy)
+- `quality`: Image quality ("standard", "hd")
+- `nb_steps`: Number of generation steps
+- `guidance_scale`: How closely to follow the prompt
+
+**Output Configuration:**
+- `nb_output`: Number of images to generate
+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
+- `output_format`: File format ("png", "jpeg", "webp")
+- `background`: Background type ("default", "transparent")
+
+**Generation Control:**
+- `seed`: Random seed (integer or "auto")
+- `is_raw`: Whether to apply post-processing
+- `is_moderated`: Enable content moderation
+- `safety_tolerance`: Content safety level (1-6)
+
+### Input Requirements
+
+PipeImgGen requires exactly one input that must be either:
+- An `ImgGenPrompt` concept
+- A concept that refines `ImgGenPrompt`
+
+The input can be named anything but must contain the prompt text for image generation.
+
+## PipeFunc operator
+
+The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+
+### Basic Usage
+
+Simple Function Call:
+```plx
+[pipe.process_data]
+type = "PipeFunc"
+definition = "Process data using custom function"
+inputs = { input_data = "DataType" }
+output = "ProcessedData"
+function_name = "process_data_function"
+```
+
+File Processing Example:
+```plx
+[pipe.read_file]
+type = "PipeFunc"
+definition = "Read file content"
+inputs = { file_path = "FilePath" }
+output = "FileContent"
+function_name = "read_file_content"
+```
+
+### Key Parameters
+
+- `function_name`: Name of the Python function to call (must be registered in func_registry)
+
+### Function Requirements
+
+The Python function must:
+
+1. **Be registered** in the `func_registry`
+2. **Accept `working_memory`** as a parameter:
+ ```python
+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
+ # Function implementation
+ pass
+ ```
+
+3. **Return appropriate types**:
+ - `StuffContent`: Single content object
+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)
+ - `str`: Simple string (becomes TextContent)
+
+### Function Registration
+
+Functions must be registered in the function registry before use:
+
+```python
+from pipelex.tools.func_registry import func_registry
+
+@func_registry.register("my_function_name")
+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
+ # Access inputs from working memory
+ input_data = working_memory.get_stuff("input_name")
+
+ # Process data
+ result = process_logic(input_data.content)
+
+ # Return result
+ return MyResultContent(data=result)
+```
+
+### Working Memory Access
+
+Inside the function, access pipeline inputs through working memory:
+
+```python
+async def process_function(working_memory: WorkingMemory) -> TextContent:
+ # Get input stuff by name
+ input_stuff = working_memory.get_stuff("input_name")
+
+ # Access the content
+ input_content = input_stuff.content
+
+ # Process and return
+ processed_text = f"Processed: {input_content.text}"
+ return TextContent(text=processed_text)
+```
+
+---
+
+## Rules to choose LLM models used in PipeLLMs.
+
+### LLM Configuration System
+
+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+
+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
+- **Routing**: `.pipelex/inference/routing_profiles.toml`
+
+### LLM Handles
+
+An llm_handle can be either:
+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+
+```toml
+[aliases]
+base-claude = "claude-4-sonnet"
+base-gpt = "gpt-5"
+base-gemini = "gemini-2.5-flash"
+base-mistral = "mistral-medium"
+```
+
+The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+
+### Using an LLM Handle in a PipeLLM
+
+Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-Example template:
```plx
+[pipe.hello_world]
+type = "PipeLLM"
+definition = "Write text about Hello World."
+output = "Text"
+llm = { llm_handle = "gpt-5", temperature = 0.9 }
prompt_template = """
-Your goal is to summarize everything related to $topic in the provided text:
+Write a haiku about Hello World.
+"""
+```
-@text
+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
+### LLM Presets
+
+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+
+Examples:
+```toml
+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }
+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```
+
+The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+
+```plx
+[pipe.extract_invoice]
+type = "PipeLLM"
+definition = "Extract invoice information from an invoice text transcript"
+inputs = { invoice_text = "InvoiceText" }
+output = "Invoice"
+llm = "llm_to_extract_invoice"
+prompt_template = """
+Extract invoice information from this invoice:
+
+The category of this invoice is: $invoice_details.category.
+
+@invoice_text
"""
```
-Here, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+
+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+
+---
-**DO NOT write "$topic" alone in an isolated line.**
-**DO write things like "Write an essay about $topic" included in an actual sentence.**
+ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
+Then, create an example file to run the pipeline in the `examples` folder.
+But don't write documentation unless asked explicitly to.
+
+---
+
+# Guide to write an example to execute a pipeline
-# Example to execute a pipeline
+## Example to execute a pipeline with text output
```python
import asyncio
from pipelex import pretty_print
-from pipelex.hub import get_pipeline_tracker, get_report_delegate
from pipelex.pipelex import Pipelex
from pipelex.pipeline.execute import execute_pipeline
-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart
+
+async def hello_world() -> str:
+ """
+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.
+ """
+ # Run the pipe
+ pipe_output = await execute_pipeline(
+ pipe_code="hello_world",
+ )
+
+ return pipe_output.main_stuff_as_str
+
+
+# start Pipelex
+Pipelex.make()
+# run sample using asyncio
+output_text = asyncio.run(hello_world())
+pretty_print(output_text, title="Your first Pipelex output")
+```
+
+## Example to execute a pipeline with structured output
+
+```python
+import asyncio
+
+from pipelex import pretty_print
+from pipelex.pipelex import Pipelex
+from pipelex.pipeline.execute import execute_pipeline
+
+from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChart
SAMPLE_NAME = "extract_gantt"
IMAGE_URL = "assets/gantt/gantt_tree_house.png"
@@ -579,22 +929,23 @@ async def extract_gantt(image_url: str) -> GanttChart:
Pipelex.make()
# run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))
-
-# Display cost report (tokens used and cost)
-get_report_delegate().generate_report()
-# output results
+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
pretty_print(gantt_chart, title="Gantt Chart")
-get_pipeline_tracker().output_flowchart()
```
-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
+## Setting up the input memory
+
+### Explanation of input memory
+
+The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
```python
StuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | str
ImplicitMemory = Dict[str, StuffContentOrData]
```
As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+### Different ways to set up the input memory
+
So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
```python
@@ -613,7 +964,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
pipe_output = await execute_pipeline(
pipe_code="power_extractor_dpe",
input_memory={
- "ocr_input": PDFContent(url=pdf_url),
+ "document": PDFContent(url=pdf_url),
},
)
@@ -654,92 +1005,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
)
```
-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
+## Using the outputs of a pipeline
-# Rules to choose LLM models used in PipeLLMs.
+All pipe executions return a `PipeOutput` object.
+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-## LLM Configuration System
+```python
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+class PipeOutput(BaseModel):
+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+ @property
+ def main_stuff(self) -> Stuff:
+ ...
-## LLM Handles
+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
+ ...
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
+ ...
-```toml
-[aliases]
-best-claude = "claude-4.1-opus"
-best-gemini = "gemini-2.5-pro"
-best-mistral = "mistral-large"
-base-gpt = "gpt-5"
-```
+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
+ ...
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+ @property
+ def main_stuff_as_text(self) -> TextContent:
+ ...
-## Using an LLM Handle in a PipeLLM
+ @property
+ def main_stuff_as_str(self) -> str:
+ ...
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
+ @property
+ def main_stuff_as_image(self) -> ImageContent:
+ ...
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-definition = "Write text about Hello World."
-output = "Text"
-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }
-prompt_template = """
-Write a haiku about Hello World.
-"""
+ @property
+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
+ ...
+
+ @property
+ def main_stuff_as_number(self) -> NumberContent:
+ ...
+
+ @property
+ def main_stuff_as_html(self) -> HtmlContent:
+ ...
+
+ @property
+ def main_stuff_as_mermaid(self) -> MermaidContent:
+ ...
```
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
+As you can see, you can extarct any variable from the output working memory.
-## LLM Presets
+### Getting the main stuff as a specific type
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+Simple text as a string:
-Examples:
-```toml
-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }
-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```python
+result = pipe_output.main_stuff_as_str
```
+Structured object (BaseModel):
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+```python
+result = pipe_output.main_stuff_as(content_type=GanttChart)
+```
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-definition = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-llm = "llm_to_extract_invoice"
-prompt_template = """
-Extract invoice information from this invoice:
+If it's a list, you can get a `ListContent` of the specific type.
-The category of this invoice is: $invoice_details.category.
+```python
+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
+```
-@invoice_text
-"""
+or if you want, you can get the actual items as a regular python list:
+
+```python
+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
```
-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+---
+# Writing unit tests
-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.
+## Unit test generalities
-These rules apply when writing unit tests.
-- Always use pytest
+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
-## Test file structure
+### Test file structure
- Name test files with `test_` prefix
- Use descriptive names that match the functionality being tested
@@ -753,7 +1108,7 @@ These rules apply when writing unit tests.
- Always put test inside Test classes.
- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
-## Markers
+### Markers
Apply the appropriate markers:
- "llm: uses an LLM to generate text or objects"
@@ -763,11 +1118,11 @@ Apply the appropriate markers:
Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-## Tips
+### Important rules
-- Never use the unittest.mock. Use pytest-mock
+- Never use the unittest.mock. Use pytest-mock.
-## Test Class Structure
+### Test Class Structure
Always group the tests of a module into a test class:
@@ -794,9 +1149,9 @@ class TestFooBar:
Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-# Pipe tests
+## Writing integration test to test pipes
-## Required imports for pipe tests
+### Required imports for pipe tests
```python
import pytest
@@ -815,7 +1170,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParams
from pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol
```
-## Pipe test implementation steps
+### Pipe test implementation steps
1. Create Stuff from blueprint:
@@ -845,14 +1200,7 @@ pipe_output = await pipe_router.run_pipe(
)
```
-4. Log output and generate report:
-
-```python
-pretty_print(pipe_output, title=f"Pipe output")
-get_report_delegate().generate_report()
-```
-
-5. Basic assertions:
+4. Basic assertions:
```python
assert pipe_output is not None
@@ -860,7 +1208,7 @@ assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
```
-## Test Data Organization
+### Test Data Organization
- If it's not already there, create a `test_data.py` file in the test directory
- Define test cases using `StuffBlueprint`:
@@ -897,28 +1245,3 @@ Also note that we provide a topic for the test case, which is purely for conveni
- Include docstrings explaining test purpose
- Log outputs for debugging
- Generate reports for cost tracking
-
-# Test-Driven Development Guide
-
-This document outlines our test-driven development (TDD) process and the tools available for testing.
-
-## TDD Cycle
-
-1. **Write a Test First**
-[pytest.mdc](pytest.mdc)
-
-2. **Write the Code**
- - Implement the minimum amount of code needed to pass the test
- - Follow the project's coding standards
- - Keep it simple - don't write more than needed
-
-3. **Run Linting and Type Checking**
-[coding_standards.mdc](coding_standards.mdc)
-
-4. **Refactor if needed**
-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)
-
-5. **Validate tests**
-
-Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.
-
diff --git a/AGENTS.md b/AGENTS.md
index c3d2602db..57111a045 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,4 +1,3 @@
-Concatenation
# Coding Standards & Best Practices
This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co
## Type Hints
1. **Always Use Type Hints**
- - Every function parameter must be typed
- - Every function return must be typed
- - Use type hints for all variables where type is not obvious
- - Use types with Uppercase first letter (Dict[], List[], etc.)
+
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals
+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`
+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
2. **StrEnum**
- Import from `pipelex.types`:
@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality co
from pipelex.types import StrEnum
```
-## BaseModel Standards
-
-- Respect Pydantic v2 standards
-- Keep models focused and single-purpose
-- Use descriptive field names
-- Use type hints for all fields
-- Document complex validations
-- Use Optional[] for nullable fields
-- Use Field(default_factory=...) for mutable defaults
+3. **Self type**
+ - Import from `pipelex.types`:
+ ```python
+ from pipelex.types import Self
+ ```
## Factory Pattern
-- Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
+
+## Error Handling
+
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones
+ - Never catch Exception, only catch specific exceptions
+ - Always add `from exc` to the exception
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
+
+ **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
## Documentation
@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co
"""
```
-## Error Handling
-
-1. **Graceful Error Handling**
- - Use try/except blocks with specific exceptions
- - Convert third-party exceptions to custom ones
- ```python
- try:
- from fal_client import AsyncClient as FalAsyncClient
- except ImportError as exc:
- raise MissingDependencyError(
- "fal-client", "fal",
- "The fal-client SDK is required to use FAL models."
- ) from exc
- ```
-
## Code Quality Checks
### Linting and Type Checking
@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.
- **Pipelines**: `pipelex/libraries/pipelines/`
- **Tests**: `tests/` directory
- **Documentation**: `docs/` directory
-# Pipeline Guide
+
+---
+
+# Guide to write or edit pipelines using the Pipelex language in .plx files
- Always first write your "plan" in natural langage, then transcribe it in pipelex.
-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)
-# Pipeline Structure Guide
-
## Pipeline File Naming
- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
- Files must be `.py` for structures
@@ -183,11 +189,11 @@ definition = "....."
```
The pipes will all have at least this base structure.
-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).
+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['ocr_input']'``
-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.
+variable='['invoice']'``
+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
<NEVER DO THIS>
@@ -201,9 +207,9 @@ inputs = {
- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-# Structured Models Rules
+## Structuring Models
-## Model Location and Registration
+### Model Location and Registration
- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`
- Models must inherit from `StructuredContent` or appropriate content type
@@ -254,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent
# Date fields should remove timezone
date_field: Optional[datetime] = None
```
-## Usage
+### Usage
Structures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.
Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.
-## Best Practices for structures
+### Best Practices for structures
- Respect Pydantic v2 standards
- Use type hints for all fields
- Use `Field` declaration and write the description
-## Pipe Controllers and Pipe Operator
+## Pipe Controllers and Pipe Operators
Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
@@ -276,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ
- `PipeSequence` - For creating a sequence of multiple steps
- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- `PipeParallel` - For parallelizing pipes
- - `PipeBatch` - For running pipes in Batch over a ListContent
2. **Operators** - For specific tasks:
- `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeOcr` - OCR Pipe
+ - `PipeOcr` - Extract text and images from an image or a PDF
+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- `PipeImgGen` - Generate Images
- `PipeFunc` - For running classic python scripts
-# PipeSequence Guide
+## PipeSequence controller
-## Purpose
-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.
+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-## Basic Structure
+### Basic Structure
```plx
[pipe.your_sequence_name]
type = "PipeSequence"
@@ -303,13 +308,13 @@ steps = [
]
```
-## Key Components
+### Key Components
1. **Steps Array**: List of pipes to execute in sequence
- `pipe`: Name of the pipe to execute
- `result`: Name to assign to the pipe's output that will be in the working memory
-## Using PipeBatch in Steps
+### Using PipeBatch in Steps
You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
@@ -330,13 +335,11 @@ steps = [
The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-# PipeCondition Controller
+## PipeCondition controller
The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-## Usage in PLX Configuration
-
-### Basic Usage with Direct Expression
+### Basic usage
```plx
[pipe.conditional_operation]
@@ -366,7 +369,7 @@ medium = "process_medium"
large = "process_large"
```
-## Key Parameters
+### Key Parameters
- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
@@ -374,41 +377,15 @@ large = "process_large"
1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.
2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger
-# PipeBatch Controller
-
-The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.
-
-## Usage in PLX Configuration
-
-```plx
-[pipe.sequence_with_batch]
-type = "PipeSequence"
-definition = "A Sequence of pipes"
-inputs = { input_data = "ConceptName" }
-output = "OutputConceptName"
-steps = [
- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }
-]
-```
-
-## Key Parameters
-
-- `pipe`: The pipe operation to apply to each element in the batch
-- `batch_over`: The name of the list in the context to iterate over
-- `batch_as`: The name to use for the current element in the pipe's context
-- `result`: Where to store the results of the batch operation
-
-# PipeLLM Guide
-
-## Purpose
+## PipeLLM operator
PipeLLM is used to:
1. Generate text or objects with LLMs
2. Process images with Vision LLMs
-## Basic Usage
+### Basic Usage
-### Simple Text Generation
+Simple Text Generation:
```plx
[pipe.write_story]
type = "PipeLLM"
@@ -419,7 +396,7 @@ Write a short story about a programmer.
"""
```
-### Structured Data Extraction
+Structured Data Extraction:
```plx
[pipe.extract_info]
type = "PipeLLM"
@@ -432,8 +409,7 @@ Extract person information from this text:
"""
```
-### System Prompts
-Add system-level instructions:
+Supports system instructions:
```plx
[pipe.expert_analysis]
type = "PipeLLM"
@@ -444,46 +420,95 @@ prompt_template = "Analyze this data"
```
### Multiple Outputs
-Generate multiple results:
+
+Generate multiple outputs (fixed number):
```plx
[pipe.generate_ideas]
type = "PipeLLM"
definition = "Generate ideas"
output = "Idea"
nb_output = 3 # Generate exactly 3 ideas
-# OR
+```
+
+Generate multiple outputs (variable number):
+```plx
+[pipe.generate_ideas]
+type = "PipeLLM"
+definition = "Generate ideas"
+output = "Idea"
multiple_output = true # Let the LLM decide how many to generate
```
-### Vision Tasks
+### Vision
+
Process images with VLMs:
```plx
[pipe.analyze_image]
type = "PipeLLM"
definition = "Analyze image"
-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }
+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }
output = "ImageAnalysis"
prompt_template = "Describe what you see in this image"
```
-# PipeOCR Guide
+### Writing prompts for PipeLLM
+
+**Insert stuff inside a tagged block**
+
+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+
+Example template:
+```plx
+prompt_template = """
+Match the expense with its corresponding invoice:
+
+@expense
+
+@invoices
+"""
+```
+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
+
+DO NOT write things like "Here is the expense: @expense".
+DO write simply "@expense" alone in an isolated line.
+
+**Insert stuff inline**
+
+If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+
+Example template:
+```plx
+prompt_template = """
+Your goal is to summarize everything related to $topic in the provided text:
+
+@text
+
+Please provide only the summary, with no additional text or explanations.
+Your summary should not be longer than 2 sentences.
+"""
+```
+
+In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
+Be sure to make the proper choice of prefix for each insertion.
+
+DO NOT write "$topic" alone in an isolated line.
+DO write things like "Write an essay about $topic" to include text into an actual sentence.
-## Purpose
-Extract text and images from an image or a PDF
+## PipeOcr operator
-## Basic Usage
+The PipeOcr operator is used to extract text and images from an image or a PDF
-### Simple Text Generation
+### Simple Text Extraction
```plx
[pipe.extract_info]
type = "PipeOcr"
definition = "extract the information"
-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input
+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
output = "Page"
```
-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.
+Only one input is allowed and it must either be an `Image` or a `PDF`.
The output concept `Page` is a native concept, with the structure `PageContent`:
It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`
@@ -500,60 +525,386 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"
- `text_and_images` are the text, and the related images found in the input image or PDF.
- `page_view` is the screenshot of the whole pdf page/image.
-This rule explains how to write prompt templates in PipeLLM definitions.
+## PipeCompose operator
-## Insert stuff inside a tagged block
+The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+### Basic Usage
-Example template:
+Simple Template Composition:
```plx
-prompt_template = """
-Match the expense with its corresponding invoice:
+[pipe.compose_report]
+type = "PipeCompose"
+definition = "Compose a report using template"
+inputs = { data = "ReportData" }
+output = "Text"
+jinja2 = """
+# Report Summary
-@expense
+Based on the analysis:
+$data
-@invoices
+Generated on: {{ current_date }}
"""
```
-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
-**DO NOT write things like "Here is the expense: @expense".**
-**DO write simply "@expense" alone in an isolated line.**
+Using Named Templates:
+```plx
+[pipe.use_template]
+type = "PipeCompose"
+definition = "Use a predefined template"
+inputs = { content = "Text" }
+output = "Text"
+jinja2_name = "standard_report_template"
+```
-## Insert stuff inline
+CRM Email Template:
+```plx
+[pipe.compose_follow_up_email]
+type = "PipeCompose"
+definition = "Compose a personalized follow-up email for CRM"
+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
+output = "Text"
+template_category = "html"
+prompting_style = { tag_style = "square_brackets", text_format = "html" }
+jinja2 = """
+Subject: Following up on our $deal.product_name discussion
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+Hi $customer.first_name,
-Example template:
+I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+
+Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+
+I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+
+**Next Steps:**
+- Schedule a demo tailored to your specific needs
+- Provide you with a customized quote based on your requirements
+- Connect you with our implementation team
+
+Would you be available for a 30-minute call this week? I have openings on:
+{% for slot in available_slots %}
+- {{ slot }}
+{% endfor %}
+
+Looking forward to moving this forward together!
+
+Best regards,
+$sales_rep.name
+$sales_rep.title
+$sales_rep.phone | $sales_rep.email
+"""
+```
+
+### Key Parameters
+
+- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)
+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)
+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
+- `prompting_style`: Styling options for template rendering
+- `extra_context`: Additional context variables for template
+
+### Template Variables
+
+Use the same variable insertion rules as PipeLLM:
+- `@variable` for block insertion (multi-line content)
+- `$variable` for inline insertion (short text)
+
+## PipeImgGen operator
+
+The PipeImgGen operator is used to generate images using AI image generation models.
+
+### Basic Usage
+
+Simple Image Generation:
+```plx
+[pipe.generate_image]
+type = "PipeImgGen"
+definition = "Generate an image from prompt"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+```
+
+Using Image Generation Settings:
+```plx
+[pipe.generate_photo]
+type = "PipeImgGen"
+definition = "Generate a high-quality photo"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Photo"
+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }
+aspect_ratio = "16:9"
+nb_steps = 8
+```
+
+Multiple Image Generation:
+```plx
+[pipe.generate_variations]
+type = "PipeImgGen"
+definition = "Generate multiple image variations"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+nb_output = 3
+seed = "auto"
+```
+
+Advanced Configuration:
```plx
+[pipe.generate_custom]
+type = "PipeImgGen"
+definition = "Generate image with custom settings"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+img_gen = "img_gen_preset_name" # Use predefined preset
+aspect_ratio = "1:1"
+quality = "hd"
+background = "transparent"
+output_format = "png"
+is_raw = false
+safety_tolerance = 3
+```
+
+### Key Parameters
+
+**Image Generation Settings:**
+- `img_gen`: ImgGenChoice (preset name or inline settings)
+- `img_gen_handle`: Direct model handle (legacy)
+- `quality`: Image quality ("standard", "hd")
+- `nb_steps`: Number of generation steps
+- `guidance_scale`: How closely to follow the prompt
+
+**Output Configuration:**
+- `nb_output`: Number of images to generate
+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
+- `output_format`: File format ("png", "jpeg", "webp")
+- `background`: Background type ("default", "transparent")
+
+**Generation Control:**
+- `seed`: Random seed (integer or "auto")
+- `is_raw`: Whether to apply post-processing
+- `is_moderated`: Enable content moderation
+- `safety_tolerance`: Content safety level (1-6)
+
+### Input Requirements
+
+PipeImgGen requires exactly one input that must be either:
+- An `ImgGenPrompt` concept
+- A concept that refines `ImgGenPrompt`
+
+The input can be named anything but must contain the prompt text for image generation.
+
+## PipeFunc operator
+
+The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+
+### Basic Usage
+
+Simple Function Call:
+```plx
+[pipe.process_data]
+type = "PipeFunc"
+definition = "Process data using custom function"
+inputs = { input_data = "DataType" }
+output = "ProcessedData"
+function_name = "process_data_function"
+```
+
+File Processing Example:
+```plx
+[pipe.read_file]
+type = "PipeFunc"
+definition = "Read file content"
+inputs = { file_path = "FilePath" }
+output = "FileContent"
+function_name = "read_file_content"
+```
+
+### Key Parameters
+
+- `function_name`: Name of the Python function to call (must be registered in func_registry)
+
+### Function Requirements
+
+The Python function must:
+
+1. **Be registered** in the `func_registry`
+2. **Accept `working_memory`** as a parameter:
+ ```python
+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
+ # Function implementation
+ pass
+ ```
+
+3. **Return appropriate types**:
+ - `StuffContent`: Single content object
+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)
+ - `str`: Simple string (becomes TextContent)
+
+### Function Registration
+
+Functions must be registered in the function registry before use:
+
+```python
+from pipelex.tools.func_registry import func_registry
+
+@func_registry.register("my_function_name")
+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
+ # Access inputs from working memory
+ input_data = working_memory.get_stuff("input_name")
+
+ # Process data
+ result = process_logic(input_data.content)
+
+ # Return result
+ return MyResultContent(data=result)
+```
+
+### Working Memory Access
+
+Inside the function, access pipeline inputs through working memory:
+
+```python
+async def process_function(working_memory: WorkingMemory) -> TextContent:
+ # Get input stuff by name
+ input_stuff = working_memory.get_stuff("input_name")
+
+ # Access the content
+ input_content = input_stuff.content
+
+ # Process and return
+ processed_text = f"Processed: {input_content.text}"
+ return TextContent(text=processed_text)
+```
+
+---
+
+## Rules to choose LLM models used in PipeLLMs.
+
+### LLM Configuration System
+
+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+
+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
+- **Routing**: `.pipelex/inference/routing_profiles.toml`
+
+### LLM Handles
+
+An llm_handle can be either:
+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+
+```toml
+[aliases]
+base-claude = "claude-4-sonnet"
+base-gpt = "gpt-5"
+base-gemini = "gemini-2.5-flash"
+base-mistral = "mistral-medium"
+```
+
+The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+
+### Using an LLM Handle in a PipeLLM
+
+Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
+
+```plx
+[pipe.hello_world]
+type = "PipeLLM"
+definition = "Write text about Hello World."
+output = "Text"
+llm = { llm_handle = "gpt-5", temperature = 0.9 }
prompt_template = """
-Your goal is to summarize everything related to $topic in the provided text:
+Write a haiku about Hello World.
+"""
+```
-@text
+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
+### LLM Presets
+
+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+
+Examples:
+```toml
+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }
+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```
+
+The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+
+```plx
+[pipe.extract_invoice]
+type = "PipeLLM"
+definition = "Extract invoice information from an invoice text transcript"
+inputs = { invoice_text = "InvoiceText" }
+output = "Invoice"
+llm = "llm_to_extract_invoice"
+prompt_template = """
+Extract invoice information from this invoice:
+
+The category of this invoice is: $invoice_details.category.
+
+@invoice_text
"""
```
-Here, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+
+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+
+---
+
+ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
+Then, create an example file to run the pipeline in the `examples` folder.
+But don't write documentation unless asked explicitly to.
+
+---
+
+# Guide to write an example to execute a pipeline
+
+## Example to execute a pipeline with text output
+
+```python
+import asyncio
+
+from pipelex import pretty_print
+from pipelex.pipelex import Pipelex
+from pipelex.pipeline.execute import execute_pipeline
+
+
+async def hello_world() -> str:
+ """
+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.
+ """
+ # Run the pipe
+ pipe_output = await execute_pipeline(
+ pipe_code="hello_world",
+ )
+
+ return pipe_output.main_stuff_as_str
-**DO NOT write "$topic" alone in an isolated line.**
-**DO write things like "Write an essay about $topic" included in an actual sentence.**
-# Example to execute a pipeline
+# start Pipelex
+Pipelex.make()
+# run sample using asyncio
+output_text = asyncio.run(hello_world())
+pretty_print(output_text, title="Your first Pipelex output")
+```
+
+## Example to execute a pipeline with structured output
```python
import asyncio
from pipelex import pretty_print
-from pipelex.hub import get_pipeline_tracker, get_report_delegate
from pipelex.pipelex import Pipelex
from pipelex.pipeline.execute import execute_pipeline
-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart
+from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChart
SAMPLE_NAME = "extract_gantt"
IMAGE_URL = "assets/gantt/gantt_tree_house.png"
@@ -578,22 +929,23 @@ async def extract_gantt(image_url: str) -> GanttChart:
Pipelex.make()
# run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))
-
-# Display cost report (tokens used and cost)
-get_report_delegate().generate_report()
-# output results
+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
pretty_print(gantt_chart, title="Gantt Chart")
-get_pipeline_tracker().output_flowchart()
```
-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
+## Setting up the input memory
+
+### Explanation of input memory
+
+The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
```python
StuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | str
ImplicitMemory = Dict[str, StuffContentOrData]
```
As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+### Different ways to set up the input memory
+
So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
```python
@@ -612,7 +964,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
pipe_output = await execute_pipeline(
pipe_code="power_extractor_dpe",
input_memory={
- "ocr_input": PDFContent(url=pdf_url),
+ "document": PDFContent(url=pdf_url),
},
)
@@ -653,92 +1005,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
)
```
-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
+## Using the outputs of a pipeline
-# Rules to choose LLM models used in PipeLLMs.
+All pipe executions return a `PipeOutput` object.
+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-## LLM Configuration System
+```python
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+class PipeOutput(BaseModel):
+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+ @property
+ def main_stuff(self) -> Stuff:
+ ...
-## LLM Handles
+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
+ ...
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
+ ...
-```toml
-[aliases]
-best-claude = "claude-4.1-opus"
-best-gemini = "gemini-2.5-pro"
-best-mistral = "mistral-large"
-base-gpt = "gpt-5"
-```
+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
+ ...
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+ @property
+ def main_stuff_as_text(self) -> TextContent:
+ ...
-## Using an LLM Handle in a PipeLLM
+ @property
+ def main_stuff_as_str(self) -> str:
+ ...
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
+ @property
+ def main_stuff_as_image(self) -> ImageContent:
+ ...
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-definition = "Write text about Hello World."
-output = "Text"
-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }
-prompt_template = """
-Write a haiku about Hello World.
-"""
+ @property
+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
+ ...
+
+ @property
+ def main_stuff_as_number(self) -> NumberContent:
+ ...
+
+ @property
+ def main_stuff_as_html(self) -> HtmlContent:
+ ...
+
+ @property
+ def main_stuff_as_mermaid(self) -> MermaidContent:
+ ...
```
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
+As you can see, you can extarct any variable from the output working memory.
-## LLM Presets
+### Getting the main stuff as a specific type
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+Simple text as a string:
-Examples:
-```toml
-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }
-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```python
+result = pipe_output.main_stuff_as_str
```
+Structured object (BaseModel):
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+```python
+result = pipe_output.main_stuff_as(content_type=GanttChart)
+```
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-definition = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-llm = "llm_to_extract_invoice"
-prompt_template = """
-Extract invoice information from this invoice:
+If it's a list, you can get a `ListContent` of the specific type.
-The category of this invoice is: $invoice_details.category.
+```python
+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
+```
-@invoice_text
-"""
+or if you want, you can get the actual items as a regular python list:
+
+```python
+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
```
-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+---
+# Writing unit tests
-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.
+## Unit test generalities
-These rules apply when writing unit tests.
-- Always use pytest
+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
-## Test file structure
+### Test file structure
- Name test files with `test_` prefix
- Use descriptive names that match the functionality being tested
@@ -752,7 +1108,7 @@ These rules apply when writing unit tests.
- Always put test inside Test classes.
- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
-## Markers
+### Markers
Apply the appropriate markers:
- "llm: uses an LLM to generate text or objects"
@@ -762,11 +1118,11 @@ Apply the appropriate markers:
Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-## Tips
+### Important rules
-- Never use the unittest.mock. Use pytest-mock
+- Never use the unittest.mock. Use pytest-mock.
-## Test Class Structure
+### Test Class Structure
Always group the tests of a module into a test class:
@@ -793,9 +1149,9 @@ class TestFooBar:
Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-# Pipe tests
+## Writing integration test to test pipes
-## Required imports for pipe tests
+### Required imports for pipe tests
```python
import pytest
@@ -814,7 +1170,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParams
from pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol
```
-## Pipe test implementation steps
+### Pipe test implementation steps
1. Create Stuff from blueprint:
@@ -844,14 +1200,7 @@ pipe_output = await pipe_router.run_pipe(
)
```
-4. Log output and generate report:
-
-```python
-pretty_print(pipe_output, title=f"Pipe output")
-get_report_delegate().generate_report()
-```
-
-5. Basic assertions:
+4. Basic assertions:
```python
assert pipe_output is not None
@@ -859,7 +1208,7 @@ assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
```
-## Test Data Organization
+### Test Data Organization
- If it's not already there, create a `test_data.py` file in the test directory
- Define test cases using `StuffBlueprint`:
@@ -896,28 +1245,3 @@ Also note that we provide a topic for the test case, which is purely for conveni
- Include docstrings explaining test purpose
- Log outputs for debugging
- Generate reports for cost tracking
-
-# Test-Driven Development Guide
-
-This document outlines our test-driven development (TDD) process and the tools available for testing.
-
-## TDD Cycle
-
-1. **Write a Test First**
-[pytest.mdc](pytest.mdc)
-
-2. **Write the Code**
- - Implement the minimum amount of code needed to pass the test
- - Follow the project's coding standards
- - Keep it simple - don't write more than needed
-
-3. **Run Linting and Type Checking**
-[coding_standards.mdc](coding_standards.mdc)
-
-4. **Refactor if needed**
-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)
-
-5. **Validate tests**
-
-Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.
-
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ca2cbc952..95b03a0cf 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -430,7 +430,6 @@ Simplified input memory:
### Changed
-- **OCR Input Standardization**: Changed OCR pipe input parameter naming to consistently use `ocr_input` for both image and PDF inputs, improving consistency across the API
- **Error Message Improvements**: Updated PipeCondition error messages to reference `expression_template` instead of deprecated `expression_jinja2`
## [v0.4.11] - 2025-06-30
diff --git a/CLAUDE.md b/CLAUDE.md
index c04686c77..57111a045 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1,4 +1,3 @@
-Concatenation
# Coding Standards & Best Practices
This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co
## Type Hints
1. **Always Use Type Hints**
- - Every function parameter must be typed
- - Every function return must be typed
- - Use type hints for all variables where type is not obvious
- - Use types with Uppercase first letter (Dict[], List[], etc.)
+
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals
+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`
+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
2. **StrEnum**
- Import from `pipelex.types`:
@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality co
from pipelex.types import StrEnum
```
-## BaseModel Standards
-
-- Respect Pydantic v2 standards
-- Keep models focused and single-purpose
-- Use descriptive field names
-- Use type hints for all fields
-- Document complex validations
-- Use Optional[] for nullable fields
-- Use Field(default_factory=...) for mutable defaults
+3. **Self type**
+ - Import from `pipelex.types`:
+ ```python
+ from pipelex.types import Self
+ ```
## Factory Pattern
-- Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
+
+## Error Handling
+
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones
+ - Never catch Exception, only catch specific exceptions
+ - Always add `from exc` to the exception
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
+
+ **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
## Documentation
@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co
"""
```
-## Error Handling
-
-1. **Graceful Error Handling**
- - Use try/except blocks with specific exceptions
- - Convert third-party exceptions to custom ones
- ```python
- try:
- from fal_client import AsyncClient as FalAsyncClient
- except ImportError as exc:
- raise MissingDependencyError(
- "fal-client", "fal",
- "The fal-client SDK is required to use FAL models."
- ) from exc
- ```
-
## Code Quality Checks
### Linting and Type Checking
@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.
- **Pipelines**: `pipelex/libraries/pipelines/`
- **Tests**: `tests/` directory
- **Documentation**: `docs/` directory
-# Pipeline Guide
+
+---
+
+# Guide to write or edit pipelines using the Pipelex language in .plx files
- Always first write your "plan" in natural langage, then transcribe it in pipelex.
-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)
-# Pipeline Structure Guide
-
## Pipeline File Naming
- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
- Files must be `.py` for structures
@@ -183,11 +189,11 @@ definition = "....."
```
The pipes will all have at least this base structure.
-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).
+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).
So If you have this error:
`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['ocr_input']'``
-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.
+variable='['invoice']'``
+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
<NEVER DO THIS>
@@ -201,9 +207,9 @@ inputs = {
- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-# Structured Models Rules
+## Structuring Models
-## Model Location and Registration
+### Model Location and Registration
- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`
- Models must inherit from `StructuredContent` or appropriate content type
@@ -227,7 +233,6 @@ If a concept only refines a native concept (like Text, Image, etc.) without addi
[concept]
Joke = "A humorous text that makes people laugh."
```
-
If you simply need to refine another native concept, construct it like this:
```plx
[concept.Landscape]
@@ -255,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent
# Date fields should remove timezone
date_field: Optional[datetime] = None
```
-## Usage
+### Usage
Structures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.
Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.
-## Best Practices for structures
+### Best Practices for structures
- Respect Pydantic v2 standards
- Use type hints for all fields
- Use `Field` declaration and write the description
-## Pipe Controllers and Pipe Operator
+## Pipe Controllers and Pipe Operators
Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
@@ -277,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ
- `PipeSequence` - For creating a sequence of multiple steps
- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- `PipeParallel` - For parallelizing pipes
- - `PipeBatch` - For running pipes in Batch over a ListContent
2. **Operators** - For specific tasks:
- `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeOcr` - OCR Pipe
+ - `PipeOcr` - Extract text and images from an image or a PDF
+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- `PipeImgGen` - Generate Images
- `PipeFunc` - For running classic python scripts
-# PipeSequence Guide
+## PipeSequence controller
-## Purpose
-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.
+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-## Basic Structure
+### Basic Structure
```plx
[pipe.your_sequence_name]
type = "PipeSequence"
@@ -304,13 +308,13 @@ steps = [
]
```
-## Key Components
+### Key Components
1. **Steps Array**: List of pipes to execute in sequence
- `pipe`: Name of the pipe to execute
- `result`: Name to assign to the pipe's output that will be in the working memory
-## Using PipeBatch in Steps
+### Using PipeBatch in Steps
You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
@@ -331,13 +335,11 @@ steps = [
The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-# PipeCondition Controller
+## PipeCondition controller
The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-## Usage in PLX Configuration
-
-### Basic Usage with Direct Expression
+### Basic usage
```plx
[pipe.conditional_operation]
@@ -367,7 +369,7 @@ medium = "process_medium"
large = "process_large"
```
-## Key Parameters
+### Key Parameters
- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
@@ -375,41 +377,15 @@ large = "process_large"
1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.
2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger
-# PipeBatch Controller
-
-The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.
-
-## Usage in PLX Configuration
-
-```plx
-[pipe.sequence_with_batch]
-type = "PipeSequence"
-definition = "A Sequence of pipes"
-inputs = { input_data = "ConceptName" }
-output = "OutputConceptName"
-steps = [
- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }
-]
-```
-
-## Key Parameters
-
-- `pipe`: The pipe operation to apply to each element in the batch
-- `batch_over`: The name of the list in the context to iterate over
-- `batch_as`: The name to use for the current element in the pipe's context
-- `result`: Where to store the results of the batch operation
-
-# PipeLLM Guide
-
-## Purpose
+## PipeLLM operator
PipeLLM is used to:
1. Generate text or objects with LLMs
2. Process images with Vision LLMs
-## Basic Usage
+### Basic Usage
-### Simple Text Generation
+Simple Text Generation:
```plx
[pipe.write_story]
type = "PipeLLM"
@@ -420,7 +396,7 @@ Write a short story about a programmer.
"""
```
-### Structured Data Extraction
+Structured Data Extraction:
```plx
[pipe.extract_info]
type = "PipeLLM"
@@ -433,8 +409,7 @@ Extract person information from this text:
"""
```
-### System Prompts
-Add system-level instructions:
+Supports system instructions:
```plx
[pipe.expert_analysis]
type = "PipeLLM"
@@ -445,46 +420,95 @@ prompt_template = "Analyze this data"
```
### Multiple Outputs
-Generate multiple results:
+
+Generate multiple outputs (fixed number):
```plx
[pipe.generate_ideas]
type = "PipeLLM"
definition = "Generate ideas"
output = "Idea"
nb_output = 3 # Generate exactly 3 ideas
-# OR
+```
+
+Generate multiple outputs (variable number):
+```plx
+[pipe.generate_ideas]
+type = "PipeLLM"
+definition = "Generate ideas"
+output = "Idea"
multiple_output = true # Let the LLM decide how many to generate
```
-### Vision Tasks
+### Vision
+
Process images with VLMs:
```plx
[pipe.analyze_image]
type = "PipeLLM"
definition = "Analyze image"
-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }
+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }
output = "ImageAnalysis"
prompt_template = "Describe what you see in this image"
```
-# PipeOCR Guide
+### Writing prompts for PipeLLM
+
+**Insert stuff inside a tagged block**
+
+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+
+Example template:
+```plx
+prompt_template = """
+Match the expense with its corresponding invoice:
+
+@expense
+
+@invoices
+"""
+```
+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
+
+DO NOT write things like "Here is the expense: @expense".
+DO write simply "@expense" alone in an isolated line.
-## Purpose
+**Insert stuff inline**
-Extract text and images from an image or a PDF
+If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+
+Example template:
+```plx
+prompt_template = """
+Your goal is to summarize everything related to $topic in the provided text:
+
+@text
+
+Please provide only the summary, with no additional text or explanations.
+Your summary should not be longer than 2 sentences.
+"""
+```
+
+In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
+Be sure to make the proper choice of prefix for each insertion.
-## Basic Usage
+DO NOT write "$topic" alone in an isolated line.
+DO write things like "Write an essay about $topic" to include text into an actual sentence.
-### Simple Text Generation
+
+## PipeOcr operator
+
+The PipeOcr operator is used to extract text and images from an image or a PDF
+
+### Simple Text Extraction
```plx
[pipe.extract_info]
type = "PipeOcr"
definition = "extract the information"
-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input
+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
output = "Page"
```
-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.
+Only one input is allowed and it must either be an `Image` or a `PDF`.
The output concept `Page` is a native concept, with the structure `PageContent`:
It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`
@@ -501,60 +525,386 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"
- `text_and_images` are the text, and the related images found in the input image or PDF.
- `page_view` is the screenshot of the whole pdf page/image.
-This rule explains how to write prompt templates in PipeLLM definitions.
+## PipeCompose operator
-## Insert stuff inside a tagged block
+The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
+### Basic Usage
-Example template:
+Simple Template Composition:
```plx
-prompt_template = """
-Match the expense with its corresponding invoice:
+[pipe.compose_report]
+type = "PipeCompose"
+definition = "Compose a report using template"
+inputs = { data = "ReportData" }
+output = "Text"
+jinja2 = """
+# Report Summary
-@expense
+Based on the analysis:
+$data
-@invoices
+Generated on: {{ current_date }}
"""
```
-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.
-**DO NOT write things like "Here is the expense: @expense".**
-**DO write simply "@expense" alone in an isolated line.**
+Using Named Templates:
+```plx
+[pipe.use_template]
+type = "PipeCompose"
+definition = "Use a predefined template"
+inputs = { content = "Text" }
+output = "Text"
+jinja2_name = "standard_report_template"
+```
-## Insert stuff inline
+CRM Email Template:
+```plx
+[pipe.compose_follow_up_email]
+type = "PipeCompose"
+definition = "Compose a personalized follow-up email for CRM"
+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
+output = "Text"
+template_category = "html"
+prompting_style = { tag_style = "square_brackets", text_format = "html" }
+jinja2 = """
+Subject: Following up on our $deal.product_name discussion
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
+Hi $customer.first_name,
+
+I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+
+Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+
+I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+
+**Next Steps:**
+- Schedule a demo tailored to your specific needs
+- Provide you with a customized quote based on your requirements
+- Connect you with our implementation team
+
+Would you be available for a 30-minute call this week? I have openings on:
+{% for slot in available_slots %}
+- {{ slot }}
+{% endfor %}
+
+Looking forward to moving this forward together!
+
+Best regards,
+$sales_rep.name
+$sales_rep.title
+$sales_rep.phone | $sales_rep.email
+"""
+```
+
+### Key Parameters
+
+- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)
+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)
+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
+- `prompting_style`: Styling options for template rendering
+- `extra_context`: Additional context variables for template
+
+### Template Variables
+
+Use the same variable insertion rules as PipeLLM:
+- `@variable` for block insertion (multi-line content)
+- `$variable` for inline insertion (short text)
+
+## PipeImgGen operator
+
+The PipeImgGen operator is used to generate images using AI image generation models.
+
+### Basic Usage
+
+Simple Image Generation:
+```plx
+[pipe.generate_image]
+type = "PipeImgGen"
+definition = "Generate an image from prompt"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+```
+
+Using Image Generation Settings:
+```plx
+[pipe.generate_photo]
+type = "PipeImgGen"
+definition = "Generate a high-quality photo"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Photo"
+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }
+aspect_ratio = "16:9"
+nb_steps = 8
+```
+
+Multiple Image Generation:
+```plx
+[pipe.generate_variations]
+type = "PipeImgGen"
+definition = "Generate multiple image variations"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+nb_output = 3
+seed = "auto"
+```
+
+Advanced Configuration:
+```plx
+[pipe.generate_custom]
+type = "PipeImgGen"
+definition = "Generate image with custom settings"
+inputs = { prompt = "ImgGenPrompt" }
+output = "Image"
+img_gen = "img_gen_preset_name" # Use predefined preset
+aspect_ratio = "1:1"
+quality = "hd"
+background = "transparent"
+output_format = "png"
+is_raw = false
+safety_tolerance = 3
+```
+
+### Key Parameters
+
+**Image Generation Settings:**
+- `img_gen`: ImgGenChoice (preset name or inline settings)
+- `img_gen_handle`: Direct model handle (legacy)
+- `quality`: Image quality ("standard", "hd")
+- `nb_steps`: Number of generation steps
+- `guidance_scale`: How closely to follow the prompt
+
+**Output Configuration:**
+- `nb_output`: Number of images to generate
+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
+- `output_format`: File format ("png", "jpeg", "webp")
+- `background`: Background type ("default", "transparent")
+
+**Generation Control:**
+- `seed`: Random seed (integer or "auto")
+- `is_raw`: Whether to apply post-processing
+- `is_moderated`: Enable content moderation
+- `safety_tolerance`: Content safety level (1-6)
+
+### Input Requirements
+
+PipeImgGen requires exactly one input that must be either:
+- An `ImgGenPrompt` concept
+- A concept that refines `ImgGenPrompt`
+
+The input can be named anything but must contain the prompt text for image generation.
+
+## PipeFunc operator
+
+The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+
+### Basic Usage
+
+Simple Function Call:
+```plx
+[pipe.process_data]
+type = "PipeFunc"
+definition = "Process data using custom function"
+inputs = { input_data = "DataType" }
+output = "ProcessedData"
+function_name = "process_data_function"
+```
+
+File Processing Example:
+```plx
+[pipe.read_file]
+type = "PipeFunc"
+definition = "Read file content"
+inputs = { file_path = "FilePath" }
+output = "FileContent"
+function_name = "read_file_content"
+```
+
+### Key Parameters
+
+- `function_name`: Name of the Python function to call (must be registered in func_registry)
+
+### Function Requirements
+
+The Python function must:
+
+1. **Be registered** in the `func_registry`
+2. **Accept `working_memory`** as a parameter:
+ ```python
+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
+ # Function implementation
+ pass
+ ```
+
+3. **Return appropriate types**:
+ - `StuffContent`: Single content object
+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)
+ - `str`: Simple string (becomes TextContent)
+
+### Function Registration
+
+Functions must be registered in the function registry before use:
+
+```python
+from pipelex.tools.func_registry import func_registry
+
+@func_registry.register("my_function_name")
+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
+ # Access inputs from working memory
+ input_data = working_memory.get_stuff("input_name")
+
+ # Process data
+ result = process_logic(input_data.content)
+
+ # Return result
+ return MyResultContent(data=result)
+```
+
+### Working Memory Access
+
+Inside the function, access pipeline inputs through working memory:
+
+```python
+async def process_function(working_memory: WorkingMemory) -> TextContent:
+ # Get input stuff by name
+ input_stuff = working_memory.get_stuff("input_name")
+
+ # Access the content
+ input_content = input_stuff.content
+
+ # Process and return
+ processed_text = f"Processed: {input_content.text}"
+ return TextContent(text=processed_text)
+```
+
+---
+
+## Rules to choose LLM models used in PipeLLMs.
+
+### LLM Configuration System
+
+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+
+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
+- **Routing**: `.pipelex/inference/routing_profiles.toml`
+
+### LLM Handles
+
+An llm_handle can be either:
+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+
+```toml
+[aliases]
+base-claude = "claude-4-sonnet"
+base-gpt = "gpt-5"
+base-gemini = "gemini-2.5-flash"
+base-mistral = "mistral-medium"
+```
+
+The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+
+### Using an LLM Handle in a PipeLLM
+
+Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-Example template:
```plx
+[pipe.hello_world]
+type = "PipeLLM"
+definition = "Write text about Hello World."
+output = "Text"
+llm = { llm_handle = "gpt-5", temperature = 0.9 }
prompt_template = """
-Your goal is to summarize everything related to $topic in the provided text:
+Write a haiku about Hello World.
+"""
+```
-@text
+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
+### LLM Presets
+
+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+
+Examples:
+```toml
+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }
+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```
+
+The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+
+```plx
+[pipe.extract_invoice]
+type = "PipeLLM"
+definition = "Extract invoice information from an invoice text transcript"
+inputs = { invoice_text = "InvoiceText" }
+output = "Invoice"
+llm = "llm_to_extract_invoice"
+prompt_template = """
+Extract invoice information from this invoice:
+
+The category of this invoice is: $invoice_details.category.
+
+@invoice_text
"""
```
-Here, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+
+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+
+---
-**DO NOT write "$topic" alone in an isolated line.**
-**DO write things like "Write an essay about $topic" included in an actual sentence.**
+ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
+Then, create an example file to run the pipeline in the `examples` folder.
+But don't write documentation unless asked explicitly to.
+
+---
+
+# Guide to write an example to execute a pipeline
-# Example to execute a pipeline
+## Example to execute a pipeline with text output
```python
import asyncio
from pipelex import pretty_print
-from pipelex.hub import get_pipeline_tracker, get_report_delegate
from pipelex.pipelex import Pipelex
from pipelex.pipeline.execute import execute_pipeline
-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart
+
+async def hello_world() -> str:
+ """
+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.
+ """
+ # Run the pipe
+ pipe_output = await execute_pipeline(
+ pipe_code="hello_world",
+ )
+
+ return pipe_output.main_stuff_as_str
+
+
+# start Pipelex
+Pipelex.make()
+# run sample using asyncio
+output_text = asyncio.run(hello_world())
+pretty_print(output_text, title="Your first Pipelex output")
+```
+
+## Example to execute a pipeline with structured output
+
+```python
+import asyncio
+
+from pipelex import pretty_print
+from pipelex.pipelex import Pipelex
+from pipelex.pipeline.execute import execute_pipeline
+
+from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChart
SAMPLE_NAME = "extract_gantt"
IMAGE_URL = "assets/gantt/gantt_tree_house.png"
@@ -579,22 +929,23 @@ async def extract_gantt(image_url: str) -> GanttChart:
Pipelex.make()
# run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))
-
-# Display cost report (tokens used and cost)
-get_report_delegate().generate_report()
-# output results
+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
pretty_print(gantt_chart, title="Gantt Chart")
-get_pipeline_tracker().output_flowchart()
```
-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
+## Setting up the input memory
+
+### Explanation of input memory
+
+The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
```python
StuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | str
ImplicitMemory = Dict[str, StuffContentOrData]
```
As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+### Different ways to set up the input memory
+
So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
```python
@@ -613,7 +964,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
pipe_output = await execute_pipeline(
pipe_code="power_extractor_dpe",
input_memory={
- "ocr_input": PDFContent(url=pdf_url),
+ "document": PDFContent(url=pdf_url),
},
)
@@ -654,92 +1005,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa
)
```
-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
+## Using the outputs of a pipeline
-# Rules to choose LLM models used in PipeLLMs.
+All pipe executions return a `PipeOutput` object.
+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-## LLM Configuration System
+```python
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+class PipeOutput(BaseModel):
+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+ @property
+ def main_stuff(self) -> Stuff:
+ ...
-## LLM Handles
+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
+ ...
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
+ ...
-```toml
-[aliases]
-best-claude = "claude-4.1-opus"
-best-gemini = "gemini-2.5-pro"
-best-mistral = "mistral-large"
-base-gpt = "gpt-5"
-```
+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
+ ...
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
+ @property
+ def main_stuff_as_text(self) -> TextContent:
+ ...
-## Using an LLM Handle in a PipeLLM
+ @property
+ def main_stuff_as_str(self) -> str:
+ ...
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
+ @property
+ def main_stuff_as_image(self) -> ImageContent:
+ ...
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-definition = "Write text about Hello World."
-output = "Text"
-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }
-prompt_template = """
-Write a haiku about Hello World.
-"""
+ @property
+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
+ ...
+
+ @property
+ def main_stuff_as_number(self) -> NumberContent:
+ ...
+
+ @property
+ def main_stuff_as_html(self) -> HtmlContent:
+ ...
+
+ @property
+ def main_stuff_as_mermaid(self) -> MermaidContent:
+ ...
```
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
+As you can see, you can extarct any variable from the output working memory.
-## LLM Presets
+### Getting the main stuff as a specific type
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
+Simple text as a string:
-Examples:
-```toml
-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }
-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+```python
+result = pipe_output.main_stuff_as_str
```
+Structured object (BaseModel):
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
+```python
+result = pipe_output.main_stuff_as(content_type=GanttChart)
+```
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-definition = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-llm = "llm_to_extract_invoice"
-prompt_template = """
-Extract invoice information from this invoice:
+If it's a list, you can get a `ListContent` of the specific type.
-The category of this invoice is: $invoice_details.category.
+```python
+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
+```
-@invoice_text
-"""
+or if you want, you can get the actual items as a regular python list:
+
+```python
+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
```
-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+---
+# Writing unit tests
-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.
+## Unit test generalities
-These rules apply when writing unit tests.
-- Always use pytest
+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
-## Test file structure
+### Test file structure
- Name test files with `test_` prefix
- Use descriptive names that match the functionality being tested
@@ -753,7 +1108,7 @@ These rules apply when writing unit tests.
- Always put test inside Test classes.
- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
-## Markers
+### Markers
Apply the appropriate markers:
- "llm: uses an LLM to generate text or objects"
@@ -763,11 +1118,11 @@ Apply the appropriate markers:
Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-## Tips
+### Important rules
-- Never use the unittest.mock. Use pytest-mock
+- Never use the unittest.mock. Use pytest-mock.
-## Test Class Structure
+### Test Class Structure
Always group the tests of a module into a test class:
@@ -794,9 +1149,9 @@ class TestFooBar:
Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-# Pipe tests
+## Writing integration test to test pipes
-## Required imports for pipe tests
+### Required imports for pipe tests
```python
import pytest
@@ -815,7 +1170,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParams
from pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol
```
-## Pipe test implementation steps
+### Pipe test implementation steps
1. Create Stuff from blueprint:
@@ -845,14 +1200,7 @@ pipe_output = await pipe_router.run_pipe(
)
```
-4. Log output and generate report:
-
-```python
-pretty_print(pipe_output, title=f"Pipe output")
-get_report_delegate().generate_report()
-```
-
-5. Basic assertions:
+4. Basic assertions:
```python
assert pipe_output is not None
@@ -860,7 +1208,7 @@ assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
```
-## Test Data Organization
+### Test Data Organization
- If it's not already there, create a `test_data.py` file in the test directory
- Define test cases using `StuffBlueprint`:
@@ -897,28 +1245,3 @@ Also note that we provide a topic for the test case, which is purely for conveni
- Include docstrings explaining test purpose
- Log outputs for debugging
- Generate reports for cost tracking
-
-# Test-Driven Development Guide
-
-This document outlines our test-driven development (TDD) process and the tools available for testing.
-
-## TDD Cycle
-
-1. **Write a Test First**
-[pytest.mdc](pytest.mdc)
-
-2. **Write the Code**
- - Implement the minimum amount of code needed to pass the test
- - Follow the project's coding standards
- - Keep it simple - don't write more than needed
-
-3. **Run Linting and Type Checking**
-[coding_standards.mdc](coding_standards.mdc)
-
-4. **Refactor if needed**
-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)
-
-5. **Validate tests**
-
-Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.
-
diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeOcr.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeOcr.md
index 130a8cc69..870cc6984 100644
--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeOcr.md
+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeOcr.md
@@ -42,7 +42,7 @@ OCR presets are defined in your model deck configuration and can include paramet
| --------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -------- |
| `type` | string | The type of the pipe: `PipeOcr` | Yes |
| `definition` | string | A description of the OCR operation. | Yes |
-| `inputs` | Fixed | The input for the PipeOcr is the key `ocr_input` and the value is either of concept `Image` or `Pdf`. | Yes |
+| `inputs` | Fixed | The value is either of concept `Image` or `Pdf`. | Yes |
| `output` | string | The output concept produced by the OCR operation. | Yes |
| `page_images` | boolean | If `true`, any images found within the document pages will be extracted and included in the output. Defaults to `false`. | No |
| `page_views` | boolean | If `true`, a high-fidelity image of each page will be included in the `page_view` field. Defaults to `false`. | No |
@@ -65,7 +65,7 @@ refines = "Page"
[pipe.extract_text_from_document]
type = "PipeOcr"
definition = "Extract text from a scanned document"
-inputs = { ocr_input = "ScannedDocument" }
+inputs = { document = "ScannedDocument" }
output = "Page"
page_views = true
page_views_dpi = 200
diff --git a/pipelex/cogt/model_backends/backend_library.py b/pipelex/cogt/model_backends/backend_library.py
index d9d27249c..ba43fe88c 100644
--- a/pipelex/cogt/model_backends/backend_library.py
+++ b/pipelex/cogt/model_backends/backend_library.py
@@ -65,7 +65,7 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):
except VarNotFoundError as var_not_found_exc:
msg = (
f"Variable substitution failed due to a variable not found error in file '{backends_library_path}':"
- "\n{var_not_found_exc}\nRun mode: '{runtime_manager.run_mode}'"
+ f"\n{var_not_found_exc}\nRun mode: '{runtime_manager.run_mode}'"
)
raise InferenceBackendCredentialsError(
error_type=InferenceBackendCredentialsErrorType.VAR_NOT_FOUND,
diff --git a/pipelex/core/concepts/concept_factory.py b/pipelex/core/concepts/concept_factory.py
index b80758831..ef37afd99 100644
--- a/pipelex/core/concepts/concept_factory.py
+++ b/pipelex/core/concepts/concept_factory.py
@@ -60,10 +60,6 @@ class ConceptFactory:
structure_class_name=TextContent.__name__,
)
- @classmethod
- def construct_concept_string_with_domain(cls, domain: str, concept_code: str) -> str:
- return f"{domain}.{concept_code}"
-
@classmethod
def make(cls, concept_code: str, domain: str, definition: str, structure_class_name: str, refines: str | None = None) -> Concept:
return Concept(
@@ -104,6 +100,25 @@ class ConceptFactory:
return DomainAndConceptCode(domain=domain, concept_code=concept_string_or_code)
return DomainAndConceptCode(domain=SpecialDomain.IMPLICIT, concept_code=concept_string_or_code)
+ @classmethod
+ def make_concept_string_with_domain(cls, domain: str, concept_code: str) -> str:
+ return f"{domain}.{concept_code}"
+
+ @classmethod
+ def make_concept_string_with_domain_from_concept_string_or_code(
+ cls, domain: str, concept_sring_or_code: str, concept_codes_from_the_same_domain: list[str] | None = None
+ ) -> str:
+ input_domain_and_code = cls.make_domain_and_concept_code_from_concept_string_or_code(
+ domain=domain,
+ concept_string_or_code=concept_sring_or_code,
+ concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
+ )
+
+ return cls.make_concept_string_with_domain(
+ domain=input_domain_and_code.domain,
+ concept_code=input_domain_and_code.concept_code,
+ )
+
@classmethod
def make_refine(cls, refine: str) -> str:
if not NativeConceptManager.is_native_concept(concept_string_or_code=refine):
diff --git a/pipelex/core/concepts/concept_library.py b/pipelex/core/concepts/concept_library.py
index 2904820f7..10b016681 100644
--- a/pipelex/core/concepts/concept_library.py
+++ b/pipelex/core/concepts/concept_library.py
@@ -123,7 +123,7 @@ class ConceptLibrary(RootModel[ConceptLibraryRoot], ConceptProviderAbstract):
ConceptBlueprint.validate_concept_code(concept_code=concept_code)
for domain in search_domains:
if found_concept := self.get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(domain=domain, concept_code=concept_code),
+ concept_string=ConceptFactory.make_concept_string_with_domain(domain=domain, concept_code=concept_code),
):
return found_concept
diff --git a/pipelex/core/pipes/pipe_input.py b/pipelex/core/pipes/pipe_input.py
index 1aa6b86fe..75bab0438 100644
--- a/pipelex/core/pipes/pipe_input.py
+++ b/pipelex/core/pipes/pipe_input.py
@@ -119,3 +119,7 @@ class PipeInputSpec(RootModel[PipeInputSpecRoot]):
),
)
return the_requirements
+
+ @property
+ def nb_inputs(self) -> int:
+ return len(self.root)
diff --git a/pipelex/core/pipes/pipe_input_factory.py b/pipelex/core/pipes/pipe_input_factory.py
index 351f4006f..7f4f8123e 100644
--- a/pipelex/core/pipes/pipe_input_factory.py
+++ b/pipelex/core/pipes/pipe_input_factory.py
@@ -2,7 +2,7 @@ from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
from pipelex.core.concepts.concept_factory import ConceptFactory
from pipelex.core.pipes.pipe_input import InputRequirement, PipeInputSpec
from pipelex.core.pipes.pipe_input_blueprint import InputRequirementBlueprint
-from pipelex.hub import get_concept_provider
+from pipelex.hub import get_required_concept
class PipeInputSpecFactory:
@@ -22,21 +22,16 @@ class PipeInputSpecFactory:
if isinstance(input_requirement_blueprint, str):
input_requirement_blueprint = InputRequirementBlueprint(concept=input_requirement_blueprint)
- concept_string = input_requirement_blueprint.concept
- ConceptBlueprint.validate_concept_string_or_code(concept_string_or_code=concept_string)
- input_domain_and_code = ConceptFactory.make_domain_and_concept_code_from_concept_string_or_code(
+ concept_string_or_code = input_requirement_blueprint.concept
+ ConceptBlueprint.validate_concept_string_or_code(concept_string_or_code=concept_string_or_code)
+ concept_string_with_domain = ConceptFactory.make_concept_string_with_domain_from_concept_string_or_code(
domain=domain,
- concept_string_or_code=concept_string,
+ concept_sring_or_code=concept_string_or_code,
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
)
inputs[var_name] = InputRequirement(
- concept=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
- domain=input_domain_and_code.domain,
- concept_code=input_domain_and_code.concept_code,
- ),
- ),
+ concept=get_required_concept(concept_string=concept_string_with_domain),
multiplicity=input_requirement_blueprint.multiplicity,
)
return PipeInputSpec(root=inputs)
diff --git a/pipelex/libraries/library_manager.py b/pipelex/libraries/library_manager.py
index 5bdf0a9fa..165894425 100644
--- a/pipelex/libraries/library_manager.py
+++ b/pipelex/libraries/library_manager.py
@@ -180,7 +180,7 @@ class LibraryManager(LibraryManagerAbstract):
# Remove concepts (they may depend on domain)
if blueprint.concept is not None:
concept_codes_to_remove = [
- ConceptFactory.construct_concept_string_with_domain(domain=blueprint.domain, concept_code=concept_code)
+ ConceptFactory.make_concept_string_with_domain(domain=blueprint.domain, concept_code=concept_code)
for concept_code in blueprint.concept
]
self.concept_library.remove_concepts_by_codes(concept_codes=concept_codes_to_remove)
diff --git a/pipelex/libraries/pipelines/builder/builder.plx b/pipelex/libraries/pipelines/builder/builder.plx
index 735c95ae1..6ff6abb07 100644
--- a/pipelex/libraries/pipelines/builder/builder.plx
+++ b/pipelex/libraries/pipelines/builder/builder.plx
@@ -77,10 +77,7 @@ We have pipe controllers:
- PipeImgGen: A pipe that uses an LLM to generate an image. VERY IMPORTANT: IF YOU DECIDE TO CREATE A PIPEIMGEN, YOU ALSO HAVE TO CREATE A PIPELLM THAT WILL WRITE THE PROMPT, AND THAT NEEDS TO PRECEED THE PIPEIMGEN, based on the necessary elements.
That means that in the MAIN pipeline, the prompt should NOT be an input. It should be a step that generates the prompt.
- PipeOcr: A pipe that uses an OCR technology to extract text from an image.
-VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE NAMED "ocr_input" and it must be either an image or a pdf or a concept which refines one of them.
-Usually there is no need to rename a variable or create a new concept, just call it ocr_input, and we'll use the basic PDF concept.
-Thats mean that in the sequence, the input should be ocr_input.
-So the input of PipeOcr is {ocr_input: "PDF"} or {ocr_input: "Image"} or a concept which refines one of them.
+VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE either an image or a pdf or a concept which refines one of them.
Be very detailed, process by steps.
@@ -225,9 +222,7 @@ THERFORE, the OUTPUT OF THIS PIPELLM should be a VARIABLE NAMED "prompt" that wi
That means that in the MAIN pipeline, the prompt should NOT be an input. It should be a step that generates the prompt.
**PipeOcr**: A pipe that uses an LLM to extract text from an image.
-- The INPUTS of this pipe is only "ocr_input" and it must be either an image or a pdf or a concept which refines one of them.
-- Usually there is no need to rename a variable or create a new concept, just call it ocr_input, and we'll use the basic PDF concept. Thats mean that in the sequence, the input should be ocr_input.
-So the input of PipeOcr is {ocr_input: "PDF"} or {ocr_input: "Image"} or a concept which refines one of them.
+- The INPUTS of PipeOcr must be either an image or a pdf or a concept which refines one of them.
**PipeFunc**: A pipe that executes a custom Python function.
- function_name: Name of the Python function to call
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe.plx b/pipelex/libraries/pipelines/builder/pipe/pipe.plx
index 8e754a15b..8de8d6971 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe.plx
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe.plx
@@ -182,7 +182,7 @@ output = "PipeOcrSpec"
prompt_template = """
Return a PipeOcrSpec for this signature.
-VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE NAMED "ocr_input" and it must be either an image or a pdf or a concept which refines one of them.
+VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE either an image or a pdf or a concept which refines one of them.
Signature:
@pipe_signature
@@ -337,7 +337,7 @@ llm = "llm_to_engineer"
prompt_template = """
Fix this failing PipeOcr spec.
-VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE NAMED "ocr_input" and it must be either an image or a pdf or a concept which refines one of them.
+VERY IMPORTANT: THE INPUT OF THE PIPEOCR must be either an image or a pdf or a concept which refines one of them.
Failing pipe:
@failed_pipe.pipe
@@ -345,7 +345,7 @@ Error message:
@failed_pipe.error_message
Please provide only the corrected PipeOcrSpec. Common OCR pipe issues to fix:
-- Input must be named 'ocr_input' and be of type Image or PDF
+- Input must be of type Image or PDF
- Output should typically be Page (native concept)
- Missing or incorrect input concept types
"""
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe_ocr_spec.py b/pipelex/libraries/pipelines/builder/pipe/pipe_ocr_spec.py
index 2a377d064..8f2ce3723 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe_ocr_spec.py
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe_ocr_spec.py
@@ -5,6 +5,12 @@ from typing_extensions import override
from pipelex.libraries.pipelines.builder.pipe.pipe_signature import PipeSpec
from pipelex.pipe_operators.ocr.pipe_ocr_blueprint import PipeOcrBlueprint
+from pipelex.types import StrEnum
+
+
+class RecommendedOcrPreset(StrEnum):
+ BASE_OCR_MISTRAL = "base_ocr_mistral"
+ BASE_OCR_PYPDFIUM2 = "base_ocr_pypdfium2"
class PipeOcrSpec(PipeSpec):
@@ -14,12 +20,12 @@ class PipeOcrSpec(PipeSpec):
Supports various OCR platforms and output configurations including image detection,
caption generation, and page rendering.
- VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE NAMED "ocr_input" and it must be either an image or a pdf or a concept which refines one of them.
+ VERY IMPORTANT: THE INPUT OF THE PIPEOCR MUST BE either an image or a pdf or a concept which refines one of them.
Attributes:
the_pipe_code: Pipe code. Must be snake_case.
type: Fixed to "PipeOcr" for this pipe type.
- ocr: Needs to be "base_ocr_mistral".
+ ocr: name of the OCR model or handle or preset or setting
page_images: Whether to include detected images in the OCR output. When enabled,
extracts and returns embedded images found in documents.
page_image_captions: Whether to generate captions for detected images using AI.
@@ -29,18 +35,12 @@ class PipeOcrSpec(PipeSpec):
page_views_dpi: DPI (dots per inch) resolution for rendered page views.
Higher values provide better quality but larger file sizes.
Defaults to configuration setting.
-
- Validation Rules:
- 1. OCR model must be "base_ocr_mistral".
- 2. Boolean flags (page_images, page_image_captions, page_views) are optional.
- 3. page_views_dpi should be a positive integer when specified.
-
"""
type: Literal["PipeOcr"] = "PipeOcr"
category: Literal["PipeOperator"] = "PipeOperator"
the_pipe_code: str = Field(description="Pipe code. Must be snake_case.")
- ocr: str = "base_ocr_mistral"
+ ocr: RecommendedOcrPreset | str = Field(description="Use one of the recommended OCR presets")
page_images: bool | None = None
page_image_captions: bool | None = None
page_views: bool | None = None
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe_parallel_spec.py b/pipelex/libraries/pipelines/builder/pipe/pipe_parallel_spec.py
index f3e481410..42e63175f 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe_parallel_spec.py
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe_parallel_spec.py
@@ -4,7 +4,7 @@ from pydantic import Field, field_validator, model_validator
from typing_extensions import override
from pipelex.exceptions import PipeDefinitionError
-from pipelex.libraries.pipelines.builder.concept.concept_spec import ConceptBlueprint
+from pipelex.libraries.pipelines.builder.concept.concept_spec import ConceptSpec
from pipelex.libraries.pipelines.builder.pipe.pipe_signature import PipeSpec
from pipelex.libraries.pipelines.builder.pipe.sub_pipe_spec import SubPipeSpec
from pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprint
@@ -47,7 +47,7 @@ class PipeParallelSpec(PipeSpec):
@staticmethod
def validate_combined_output(combined_output: str) -> str:
if combined_output:
- ConceptBlueprint.validate_concept_string_or_code(concept_string_or_code=combined_output)
+ ConceptSpec.validate_concept_string_or_code(concept_string_or_code=combined_output)
return combined_output
@model_validator(mode="after")
diff --git a/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py b/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
index 8c9f667a5..e100ac393 100644
--- a/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
+++ b/pipelex/libraries/pipelines/builder/pipe/pipe_signature.py
@@ -6,7 +6,7 @@ from pipelex.core.pipes.exceptions import PipeBlueprintError
from pipelex.core.pipes.pipe_blueprint import AllowedPipeCategories, AllowedPipeTypes, PipeBlueprint
from pipelex.core.pipes.pipe_input_blueprint import InputRequirementBlueprint
from pipelex.core.stuffs.stuff_content import StructuredContent
-from pipelex.libraries.pipelines.builder.concept.concept_spec import ConceptBlueprint, ConceptSpecDraft
+from pipelex.libraries.pipelines.builder.concept.concept_spec import ConceptSpec, ConceptSpecDraft
from pipelex.libraries.pipelines.builder.pipe.inputs_spec import InputRequirementSpec
from pipelex.tools.misc.string_utils import is_snake_case
@@ -77,7 +77,7 @@ class PipeSpec(StructuredContent):
@field_validator("output", mode="before")
@staticmethod
def validate_concept_string_or_code(output: str) -> str:
- ConceptBlueprint.validate_concept_string_or_code(concept_string_or_code=output)
+ ConceptSpec.validate_concept_string_or_code(concept_string_or_code=output)
return output
@classmethod
diff --git a/pipelex/libraries/pipelines/documents.plx b/pipelex/libraries/pipelines/documents.plx
index 374b62e2c..9f1b15d1d 100644
--- a/pipelex/libraries/pipelines/documents.plx
+++ b/pipelex/libraries/pipelines/documents.plx
@@ -13,7 +13,7 @@ TextAndImagesContent = "A content that comprises text and images where the text
[pipe.ocr_page_contents_from_pdf]
type = "PipeOcr"
definition = "Extract page contents from a PDF document"
-inputs = { ocr_input = "PDF" }
+inputs = { document = "PDF" }
output = "Page"
page_images = true
page_views = false
@@ -22,7 +22,7 @@ ocr = "mistral-ocr"
[pipe.ocr_page_contents_and_views_from_pdf]
type = "PipeOcr"
definition = "Extract page contents from a PDF document as well as full page views"
-inputs = { ocr_input = "PDF" }
+inputs = { document = "PDF" }
output = "Page"
page_images = true
page_views = true
diff --git a/pipelex/pipe_controllers/batch/pipe_batch_factory.py b/pipelex/pipe_controllers/batch/pipe_batch_factory.py
index 469bccaea..699aed0b6 100644
--- a/pipelex/pipe_controllers/batch/pipe_batch_factory.py
+++ b/pipelex/pipe_controllers/batch/pipe_batch_factory.py
@@ -34,7 +34,7 @@ class PipeBatchFactory(PipeFactoryProtocol[PipeBatchBlueprint, PipeBatch]):
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_controllers/condition/pipe_condition_factory.py b/pipelex/pipe_controllers/condition/pipe_condition_factory.py
index 215c7c978..8f4382eb5 100644
--- a/pipelex/pipe_controllers/condition/pipe_condition_factory.py
+++ b/pipelex/pipe_controllers/condition/pipe_condition_factory.py
@@ -40,7 +40,7 @@ class PipeConditionFactory(PipeFactoryProtocol[PipeConditionBlueprint, PipeCondi
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_controllers/parallel/pipe_parallel_factory.py b/pipelex/pipe_controllers/parallel/pipe_parallel_factory.py
index b43fa0423..af35e31d2 100644
--- a/pipelex/pipe_controllers/parallel/pipe_parallel_factory.py
+++ b/pipelex/pipe_controllers/parallel/pipe_parallel_factory.py
@@ -46,7 +46,7 @@ class PipeParallelFactory(PipeFactoryProtocol[PipeParallelBlueprint, PipeParalle
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
)
combined_output = get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=combined_output_domain_and_code.domain,
concept_code=combined_output_domain_and_code.concept_code,
),
@@ -69,7 +69,7 @@ class PipeParallelFactory(PipeFactoryProtocol[PipeParallelBlueprint, PipeParalle
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py b/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py
index a91d084cd..dc3d42644 100644
--- a/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py
+++ b/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py
@@ -34,7 +34,7 @@ class PipeSequenceFactory(PipeFactoryProtocol[PipeSequenceBlueprint, PipeSequenc
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_operators/compose/pipe_compose_factory.py b/pipelex/pipe_operators/compose/pipe_compose_factory.py
index 4dc329138..0b739c018 100644
--- a/pipelex/pipe_operators/compose/pipe_compose_factory.py
+++ b/pipelex/pipe_operators/compose/pipe_compose_factory.py
@@ -48,7 +48,7 @@ class PipeComposeFactory(PipeFactoryProtocol[PipeComposeBlueprint, PipeCompose])
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_operators/func/pipe_func_factory.py b/pipelex/pipe_operators/func/pipe_func_factory.py
index 685423dc6..792a539e1 100644
--- a/pipelex/pipe_operators/func/pipe_func_factory.py
+++ b/pipelex/pipe_operators/func/pipe_func_factory.py
@@ -33,7 +33,7 @@ class PipeFuncFactory(PipeFactoryProtocol[PipeFuncBlueprint, PipeFunc]):
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_operators/img_gen/pipe_img_gen_factory.py b/pipelex/pipe_operators/img_gen/pipe_img_gen_factory.py
index de36dde65..6c36f3829 100644
--- a/pipelex/pipe_operators/img_gen/pipe_img_gen_factory.py
+++ b/pipelex/pipe_operators/img_gen/pipe_img_gen_factory.py
@@ -33,7 +33,7 @@ class PipeImgGenFactory(PipeFactoryProtocol[PipeImgGenBlueprint, PipeImgGen]):
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/pipe_operators/llm/pipe_llm.py b/pipelex/pipe_operators/llm/pipe_llm.py
index 3204c4503..e7478f5fe 100644
--- a/pipelex/pipe_operators/llm/pipe_llm.py
+++ b/pipelex/pipe_operators/llm/pipe_llm.py
@@ -248,7 +248,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):
output_concept_code = SpecialDomain.NATIVE + "." + NativeConceptEnum.TEXT
else:
output_concept = get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(domain=self.domain, concept_code=output_concept_code),
+ concept_string=ConceptFactory.make_concept_string_with_domain(domain=self.domain, concept_code=output_concept_code),
)
multiplicity_resolution = output_multiplicity_to_apply(
diff --git a/pipelex/pipe_operators/llm/pipe_llm_factory.py b/pipelex/pipe_operators/llm/pipe_llm_factory.py
index 60410525b..b6334debd 100644
--- a/pipelex/pipe_operators/llm/pipe_llm_factory.py
+++ b/pipelex/pipe_operators/llm/pipe_llm_factory.py
@@ -85,7 +85,7 @@ class PipeLLMFactory(PipeFactoryProtocol[PipeLLMBlueprint, PipeLLM]):
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
)
concept = get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=domain_and_code.domain,
concept_code=domain_and_code.concept_code,
),
@@ -132,7 +132,7 @@ class PipeLLMFactory(PipeFactoryProtocol[PipeLLMBlueprint, PipeLLM]):
concept_codes_from_the_same_domain=concept_codes_from_the_same_domain,
),
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(domain=output_concept_domain, concept_code=output_concept_code),
+ concept_string=ConceptFactory.make_concept_string_with_domain(domain=output_concept_domain, concept_code=output_concept_code),
),
llm_prompt_spec=llm_prompt_spec,
llm_choices=llm_choices,
diff --git a/pipelex/pipe_operators/ocr/pipe_ocr.py b/pipelex/pipe_operators/ocr/pipe_ocr.py
index 34af1cf67..1d9838d8e 100644
--- a/pipelex/pipe_operators/ocr/pipe_ocr.py
+++ b/pipelex/pipe_operators/ocr/pipe_ocr.py
@@ -14,8 +14,6 @@ from pipelex.config import StaticValidationReaction, get_config
from pipelex.core.concepts.concept_native import NativeConceptEnum
from pipelex.core.memory.working_memory import WorkingMemory
from pipelex.core.pipes.pipe_input import PipeInputSpec
-from pipelex.core.pipes.pipe_input_blueprint import InputRequirementBlueprint
-from pipelex.core.pipes.pipe_input_factory import PipeInputSpecFactory
from pipelex.core.pipes.pipe_output import PipeOutput
from pipelex.core.pipes.pipe_run_params import PipeRunMode, PipeRunParams
from pipelex.core.stuffs.stuff_content import ImageContent, ListContent, PageContent, TextAndImagesContent, TextContent
@@ -40,9 +38,6 @@ class PipeOcrOutput(PipeOutput):
pass
-PIPE_OCR_INPUT_NAME = "ocr_input"
-
-
class PipeOcr(PipeOperator[PipeOcrOutput]):
type: Literal["PipeOcr"] = "PipeOcr"
ocr_choice: OcrChoice | None
@@ -67,7 +62,7 @@ class PipeOcr(PipeOperator[PipeOcrOutput]):
@override
def required_variables(self) -> set[str]:
- return {PIPE_OCR_INPUT_NAME}
+ return set(self.inputs.required_names)
@override
def validate_output(self):
@@ -81,49 +76,14 @@ class PipeOcr(PipeOperator[PipeOcrOutput]):
default_reaction = static_validation_config.default_reaction
reactions = static_validation_config.reactions
- # check that we have either an image or a pdf in inputs, at most one of them and nothing else
- candidate_prompt_var_names: list[str] = []
- for input_name, requirement in self.inputs.items:
- log.debug(f"{input_name=}")
- log.debug(f"{requirement=}")
- log.debug(f"Validating input '{input_name}' with concept code '{requirement.concept.code}'")
- if concept_provider.is_compatible(
- tested_concept=requirement.concept,
- wanted_concept=concept_provider.get_native_concept(native_concept=NativeConceptEnum.IMAGE),
- strict=True,
- ):
- self.image_stuff_name = input_name
- candidate_prompt_var_names.append(input_name)
- elif concept_provider.is_compatible(
- tested_concept=requirement.concept,
- wanted_concept=concept_provider.get_native_concept(native_concept=NativeConceptEnum.PDF),
- strict=True,
- ):
- self.pdf_stuff_name = input_name
- candidate_prompt_var_names.append(input_name)
- else:
- inadequate_input_concept_error = StaticValidationError(
- error_type=StaticValidationErrorType.INADEQUATE_INPUT_CONCEPT,
- domain=self.domain,
- pipe_code=self.code,
- variable_names=[input_name],
- provided_concept_code=requirement.concept.code,
- explanation="For OCR you must provide either a pdf or an image or a concept that refines one of them",
- )
- match reactions.get(StaticValidationErrorType.INADEQUATE_INPUT_CONCEPT, default_reaction):
- case StaticValidationReaction.IGNORE:
- pass
- case StaticValidationReaction.LOG:
- log.error(inadequate_input_concept_error.desc())
- case StaticValidationReaction.RAISE:
- raise inadequate_input_concept_error
-
- if len(candidate_prompt_var_names) > 1:
+ # check that we have exactly one input
+ nb_inputs = self.inputs.nb_inputs
+ if nb_inputs > 1:
too_many_candidate_inputs_error = StaticValidationError(
error_type=StaticValidationErrorType.TOO_MANY_CANDIDATE_INPUTS,
domain=self.domain,
pipe_code=self.code,
- variable_names=candidate_prompt_var_names,
+ variable_names=self.inputs.variables,
explanation="Only one image or pdf can be provided for OCR",
)
match reactions.get(StaticValidationErrorType.TOO_MANY_CANDIDATE_INPUTS, default_reaction):
@@ -133,7 +93,7 @@ class PipeOcr(PipeOperator[PipeOcrOutput]):
log.error(too_many_candidate_inputs_error.desc())
case StaticValidationReaction.RAISE:
raise too_many_candidate_inputs_error
- elif len(candidate_prompt_var_names) == 0:
+ elif nb_inputs < 1:
missing_input_var_error = StaticValidationError(
error_type=StaticValidationErrorType.MISSING_INPUT_VARIABLE,
domain=self.domain,
@@ -148,12 +108,44 @@ class PipeOcr(PipeOperator[PipeOcrOutput]):
case StaticValidationReaction.RAISE:
raise missing_input_var_error
+ # We have confirmed right above that we have exactly one input
+ # get input_name, requirement from single item in inputs
+ input_name, requirement = self.inputs.items[0]
+ log.debug(f"{input_name=}")
+ log.debug(f"{requirement=}")
+ log.debug(f"Validating input '{input_name}' with concept code '{requirement.concept.code}'")
+ if concept_provider.is_compatible(
+ tested_concept=requirement.concept,
+ wanted_concept=concept_provider.get_native_concept(native_concept=NativeConceptEnum.IMAGE),
+ strict=True,
+ ):
+ self.image_stuff_name = input_name
+ elif concept_provider.is_compatible(
+ tested_concept=requirement.concept,
+ wanted_concept=concept_provider.get_native_concept(native_concept=NativeConceptEnum.PDF),
+ strict=True,
+ ):
+ self.pdf_stuff_name = input_name
+ else:
+ inadequate_input_concept_error = StaticValidationError(
+ error_type=StaticValidationErrorType.INADEQUATE_INPUT_CONCEPT,
+ domain=self.domain,
+ pipe_code=self.code,
+ variable_names=[input_name],
+ provided_concept_code=requirement.concept.code,
+ explanation="For OCR you must provide either a pdf or an image or a concept that refines one of them",
+ )
+ match reactions.get(StaticValidationErrorType.INADEQUATE_INPUT_CONCEPT, default_reaction):
+ case StaticValidationReaction.IGNORE:
+ pass
+ case StaticValidationReaction.LOG:
+ log.error(inadequate_input_concept_error.desc())
+ case StaticValidationReaction.RAISE:
+ raise inadequate_input_concept_error
+
@override
def needed_inputs(self, visited_pipes: set[str] | None = None) -> PipeInputSpec:
- return PipeInputSpecFactory.make_from_blueprint(
- domain=self.domain,
- blueprint={PIPE_OCR_INPUT_NAME: InputRequirementBlueprint(concept=self.inputs.root[PIPE_OCR_INPUT_NAME].concept.concept_string)},
- )
+ return self.inputs
@override
async def _run_operator_pipe(
diff --git a/pipelex/pipe_operators/ocr/pipe_ocr_factory.py b/pipelex/pipe_operators/ocr/pipe_ocr_factory.py
index 65fd09980..0a2c26579 100644
--- a/pipelex/pipe_operators/ocr/pipe_ocr_factory.py
+++ b/pipelex/pipe_operators/ocr/pipe_ocr_factory.py
@@ -30,7 +30,7 @@ class PipeOcrFactory(PipeFactoryProtocol[PipeOcrBlueprint, PipeOcr]):
code=pipe_code,
definition=blueprint.definition,
output=get_concept_provider().get_required_concept(
- concept_string=ConceptFactory.construct_concept_string_with_domain(
+ concept_string=ConceptFactory.make_concept_string_with_domain(
domain=output_domain_and_code.domain,
concept_code=output_domain_and_code.concept_code,
),
diff --git a/pipelex/tools/templating/template_preprocessor.py b/pipelex/tools/templating/template_preprocessor.py
index a7b82174b..7cfdadc5b 100644
--- a/pipelex/tools/templating/template_preprocessor.py
+++ b/pipelex/tools/templating/template_preprocessor.py
@@ -38,12 +38,20 @@ def _detect_non_existent_filters(template_str: str) -> None:
# Handle @variable patterns
def replace_at_variable(match: Match[str]) -> str:
variable: str = match.group(1)
+ if variable.endswith("."):
+ # trailing dot can't be in a variable name so it must be a punctuation in the template sentence, so we remove it
+ variable = variable[:-1]
+ return f'{{{{ {variable}|tag("{variable}") }}}}.'
return f'{{{{ {variable}|tag("{variable}") }}}}'
# Handle @?variable patterns (optional insertion)
def replace_optional_at_variable(match: Match[str]) -> str:
variable: str = match.group(1)
+ if variable.endswith("."):
+ # trailing dot can't be in a variable name so it must be a punctuation in the template sentence, so we remove it
+ variable = variable[:-1]
+ return f'{{% if {variable} %}}{{{{ {variable}|tag("{variable}") }}}}{{% endif %}}.'
return f'{{% if {variable} %}}{{{{ {variable}|tag("{variable}") }}}}{{% endif %}}'
diff --git a/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_validation.py b/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_validation.py
index 1c521b07d..dc8f9c07f 100644
--- a/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_validation.py
+++ b/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_validation.py
@@ -36,7 +36,7 @@ class TestPipeParallelValidation:
"context": InputRequirementBlueprint(concept=concept_2.concept_string),
},
definition="Analysis pipe for document processing",
- output=ConceptFactory.construct_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
+ output=ConceptFactory.make_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
prompt_template="Analyze this document: \n@context\n@document",
)
@@ -61,7 +61,7 @@ class TestPipeParallelValidation:
"document": InputRequirementBlueprint(concept=concept_1.code),
"context": InputRequirementBlueprint(concept=concept_2.code),
},
- output=ConceptFactory.construct_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
+ output=ConceptFactory.make_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
parallels=[SubPipeBlueprint(pipe=real_pipe.code, result="analysis_result")],
add_each_output=True,
combined_output=None,
@@ -105,7 +105,7 @@ class TestPipeParallelValidation:
pipe_parallel_blueprint = PipeParallelBlueprint(
definition="Basic parallel pipe for testing",
inputs={"input_var": InputRequirementBlueprint(concept=concept_1.concept_string)},
- output=ConceptFactory.construct_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
+ output=ConceptFactory.make_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
parallels=[SubPipeBlueprint(pipe="test_pipe_1", result="result_1")],
add_each_output=True,
combined_output=None,
@@ -152,7 +152,7 @@ class TestPipeParallelValidation:
"document": InputRequirementBlueprint(concept=concept_1.concept_string),
"context": InputRequirementBlueprint(concept=concept_2.concept_string),
},
- output=ConceptFactory.construct_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
+ output=ConceptFactory.make_concept_string_with_domain(domain=domain, concept_code=concept_3.code),
parallels=[], # No sub-pipes to avoid dependency issues
add_each_output=True,
combined_output=None,
diff --git a/tests/integration/pipelex/pipes/operator/pipe_ocr/test_pipe_ocr.py b/tests/integration/pipelex/pipes/operator/pipe_ocr/test_pipe_ocr.py
index c2961ba1b..dfba3613c 100644
--- a/tests/integration/pipelex/pipes/operator/pipe_ocr/test_pipe_ocr.py
+++ b/tests/integration/pipelex/pipes/operator/pipe_ocr/test_pipe_ocr.py
@@ -12,7 +12,6 @@ from pipelex.core.pipes.pipe_run_params import PipeRunMode
from pipelex.core.pipes.pipe_run_params_factory import PipeRunParamsFactory
from pipelex.core.stuffs.stuff_content import PageContent
from pipelex.hub import get_concept_provider, get_pipe_router
-from pipelex.pipe_operators.ocr.pipe_ocr import PIPE_OCR_INPUT_NAME
from pipelex.pipe_operators.ocr.pipe_ocr_blueprint import PipeOcrBlueprint
from pipelex.pipe_operators.ocr.pipe_ocr_factory import PipeOcrFactory
from pipelex.pipe_works.pipe_job_factory import PipeJobFactory
@@ -84,9 +83,10 @@ class TestPipeOCR:
pipe_run_mode: PipeRunMode,
pdf_url: str,
):
+ input_name = "arbitrary_name"
pipe_ocr_blueprint = PipeOcrBlueprint(
definition="OCR test for PDF processing",
- inputs={PIPE_OCR_INPUT_NAME: InputRequirementBlueprint(concept=NativeConceptEnum.PDF)},
+ inputs={input_name: InputRequirementBlueprint(concept=NativeConceptEnum.PDF)},
output=NativeConceptEnum.TEXT_AND_IMAGES,
ocr=ocr_choice_for_pdf,
page_images=True,
@@ -104,7 +104,7 @@ class TestPipeOCR:
pipe_run_params=PipeRunParamsFactory.make_run_params(pipe_run_mode=pipe_run_mode),
working_memory=WorkingMemoryFactory.make_from_pdf(
pdf_url=pdf_url,
- name=PIPE_OCR_INPUT_NAME,
+ name=input_name,
),
)
pipe_ocr_output = await get_pipe_router().run(
diff --git a/tests/integration/pipelex/test_data.py b/tests/integration/pipelex/test_data.py
index a4f22dc0a..16b415911 100644
--- a/tests/integration/pipelex/test_data.py
+++ b/tests/integration/pipelex/test_data.py
@@ -13,7 +13,6 @@ from pipelex.core.stuffs.stuff_content import (
)
from pipelex.core.stuffs.stuff_factory import StuffBlueprint, StuffFactory
from pipelex.exceptions import PipeStackOverflowError
-from pipelex.pipe_operators.ocr.pipe_ocr import PIPE_OCR_INPUT_NAME
from tests.cases import ImageTestCases, PDFTestCases
@@ -60,7 +59,7 @@ class PipeTestCases:
content=ImageContent(url=URL_IMG_FASHION_PHOTO_1),
)
SIMPLE_STUFF_PDF = StuffFactory.make_stuff(
- name=PIPE_OCR_INPUT_NAME,
+ name="document",
concept=ConceptFactory.make_native_concept(native_concept_data=NATIVE_CONCEPTS_DATA[NativeConceptEnum.PDF]),
content=PDFContent(url=PDFTestCases.DOCUMENT_URLS[0]),
)
diff --git a/tests/unit/pipelex/core/concepts/concept/test_concept.py b/tests/unit/pipelex/core/concepts/concept/test_concept.py
index de683c378..0c0cb8979 100644
--- a/tests/unit/pipelex/core/concepts/concept/test_concept.py
+++ b/tests/unit/pipelex/core/concepts/concept/test_concept.py
@@ -204,7 +204,7 @@ class TestConcept:
"""Test construct_concept_string_with_domain method."""
valid_domain = "valid_domain"
assert (
- ConceptFactory.construct_concept_string_with_domain(domain=valid_domain, concept_code=NativeConceptEnum.TEXT)
+ ConceptFactory.make_concept_string_with_domain(domain=valid_domain, concept_code=NativeConceptEnum.TEXT)
== f"{valid_domain}.{NativeConceptEnum.TEXT}"
)
diff --git a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_data.py b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_data.py
index 06786ebdc..653c80549 100644
--- a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_data.py
+++ b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_data.py
@@ -12,17 +12,17 @@ class PipeOcrTestCases:
PipeOcrSpec(
the_pipe_code="ocr_extractor",
definition="Extract text from image",
- inputs={"ocr_input": InputRequirementSpec(concept="Image")},
+ inputs={"image": InputRequirementSpec(concept="Image")},
output="ExtractedText",
- ocr="mistral-pixtral",
+ ocr="base_ocr_mistral",
),
PipeOcrBlueprint(
definition="Extract text from image",
- inputs={"ocr_input": InputRequirementBlueprint(concept="Image")},
+ inputs={"image": InputRequirementBlueprint(concept="Image")},
output="ExtractedText",
type="PipeOcr",
category="PipeOperator",
- ocr="mistral-pixtral",
+ ocr="base_ocr_mistral",
),
)
@@ -31,9 +31,9 @@ class PipeOcrTestCases:
PipeOcrSpec(
the_pipe_code="advanced_ocr",
definition="OCR with page options",
- inputs={"ocr_input": InputRequirementSpec(concept="PDF")},
+ inputs={"document": InputRequirementSpec(concept="PDF")},
output="PageContent",
- ocr="tesseract",
+ ocr="base_ocr_mistral",
page_images=True,
page_image_captions=True,
page_views=True,
@@ -41,11 +41,11 @@ class PipeOcrTestCases:
),
PipeOcrBlueprint(
definition="OCR with page options",
- inputs={"ocr_input": InputRequirementBlueprint(concept="PDF")},
+ inputs={"document": InputRequirementBlueprint(concept="PDF")},
output="PageContent",
type="PipeOcr",
category="PipeOperator",
- ocr="tesseract",
+ ocr="base_ocr_mistral",
),
)
diff --git a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_pipe_ocr.py b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_pipe_ocr.py
index f4cdfdafb..d3d63b25b 100644
--- a/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_pipe_ocr.py
+++ b/tests/unit/pipelex/libraries/pipelines/builder/pipe/pipe_operators/pipe_ocr/test_pipe_ocr.py
@@ -3,6 +3,7 @@ import pytest
from pipelex.libraries.pipelines.builder.pipe.pipe_ocr_spec import PipeOcrSpec
from pipelex.pipe_operators.ocr.pipe_ocr_blueprint import PipeOcrBlueprint
+from pipelex import log
from .test_data import PipeOcrTestCases
@@ -17,5 +18,6 @@ class TestPipeOcrBlueprintConversion:
pipe_spec: PipeOcrSpec,
expected_blueprint: PipeOcrBlueprint,
):
+ log.verbose(f"Testing {test_name}")
result = pipe_spec.to_blueprint()
assert result == expected_blueprint
diff --git a/tests/unit/pipelex/tools/templating/test_template_preprocessor.py b/tests/unit/pipelex/tools/templating/test_template_preprocessor.py
index 76f56f99b..ba7b67028 100644
--- a/tests/unit/pipelex/tools/templating/test_template_preprocessor.py
+++ b/tests/unit/pipelex/tools/templating/test_template_preprocessor.py
@@ -120,3 +120,24 @@ Optional notes:
'{% if optional_var %}{{ optional_var|tag("optional_var") }}{% endif %}'
)
assert result == expected
+
+ def test_at_variable_with_trailing_dot(self):
+ """Test @variable pattern with trailing dot (punctuation)."""
+ template = "Extract employee information from this invoice text: @invoice_text."
+ result = preprocess_template(template)
+ expected = 'Extract employee information from this invoice text: {{ invoice_text|tag("invoice_text") }}.'
+ assert result == expected
+
+ def test_optional_at_variable_with_trailing_dot(self):
+ """Test @?variable pattern with trailing dot (punctuation)."""
+ template = "Optional information: @?optional_data."
+ result = preprocess_template(template)
+ expected = 'Optional information: {% if optional_data %}{{ optional_data|tag("optional_data") }}{% endif %}.'
+ assert result == expected
+
+ def test_multiple_at_variables_with_trailing_dots(self):
+ """Test multiple @variable patterns with trailing dots."""
+ template = "Extract all articles from this invoice text: @invoice_text. Process the items: @item_list."
+ result = preprocess_template(template)
+ expected = 'Extract all articles from this invoice text: {{ invoice_text|tag("invoice_text") }}. Process the items: {{ item_list|tag("item_list") }}.'
+ assert result == expected
diff --git a/tests/unit/pipelex/tools/test_class_registry_utils.py b/tests/unit/pipelex/tools/test_class_registry_utils.py
index 59c9a873d..fb29f6405 100644
--- a/tests/unit/pipelex/tools/test_class_registry_utils.py
+++ b/tests/unit/pipelex/tools/test_class_registry_utils.py
@@ -1,10 +1,9 @@
from pathlib import Path
+from pipelex.tools.class_registry_utils import ClassRegistryUtils
from pydantic import BaseModel
from pytest_mock import MockerFixture
-from pipelex.tools.class_registry_utils import ClassRegistryUtils
-
class TestClassRegistryUtilsUnit:
"""Unit tests for ClassRegistryUtils using mocks."""
@@ -16,18 +15,31 @@ class TestClassRegistryUtilsUnit:
mock_module.__name__ = "test_module"
# Mock the functions at the location where they're imported in ClassRegistryUtils
- mock_import = mocker.patch("pipelex.tools.class_registry_utils.import_module_from_file", return_value=mock_module)
- mock_find = mocker.patch("pipelex.tools.class_registry_utils.find_classes_in_module", return_value=[str, int])
+ mock_import = mocker.patch(
+ "pipelex.tools.class_registry_utils.import_module_from_file",
+ return_value=mock_module,
+ )
+ mock_find = mocker.patch(
+ "pipelex.tools.class_registry_utils.find_classes_in_module",
+ return_value=[str, int],
+ )
# Mock the global class registry
mock_registry = mocker.MagicMock()
- mocker.patch("pipelex.tools.class_registry_utils.KajsonManager.get_class_registry", return_value=mock_registry)
+ mocker.patch(
+ "pipelex.tools.class_registry_utils.KajsonManager.get_class_registry",
+ return_value=mock_registry,
+ )
- ClassRegistryUtils.register_classes_in_file(file_path="/fake/path.py", base_class=None, is_include_imported=False)
+ ClassRegistryUtils.register_classes_in_file(
+ file_path="/fake/path.py", base_class=None, is_include_imported=False
+ )
# Verify the mocked functions were called correctly
mock_import.assert_called_once_with("/fake/path.py")
- mock_find.assert_called_once_with(module=mock_module, base_class=None, include_imported=False)
+ mock_find.assert_called_once_with(
+ module=mock_module, base_class=None, include_imported=False
+ )
# Verify classes were registered with the global registry
mock_registry.register_classes.assert_called_once_with(classes=[str, int])
@@ -36,15 +48,30 @@ class TestClassRegistryUtilsUnit:
"""Unit test for registering classes from a folder using mocks."""
# Mock the file finding and registration
mock_files = [Path("/fake/file1.py"), Path("/fake/file2.py")]
- mock_find_files = mocker.patch.object(ClassRegistryUtils, "find_files_in_dir", return_value=mock_files)
- mock_register_file = mocker.patch.object(ClassRegistryUtils, "register_classes_in_file")
+ mock_find_files = mocker.patch.object(
+ ClassRegistryUtils, "find_files_in_dir", return_value=mock_files
+ )
+ mock_register_file = mocker.patch.object(
+ ClassRegistryUtils, "register_classes_in_file"
+ )
- ClassRegistryUtils.register_classes_in_folder(folder_path="/fake/folder", base_class=BaseModel, is_recursive=True, is_include_imported=False)
+ ClassRegistryUtils.register_classes_in_folder(
+ folder_path="/fake/folder",
+ base_class=BaseModel,
+ is_recursive=True,
+ is_include_imported=False,
+ )
# Verify find_files_in_dir was called correctly
- mock_find_files.assert_called_once_with(dir_path="/fake/folder", pattern="*.py", is_recursive=True)
+ mock_find_files.assert_called_once_with(
+ dir_path="/fake/folder", pattern="*.py", is_recursive=True
+ )
# Verify register_classes_in_file was called for each file
assert mock_register_file.call_count == 2
- mock_register_file.assert_any_call(file_path="/fake/file1.py", base_class=BaseModel, is_include_imported=False)
- mock_register_file.assert_any_call(file_path="/fake/file2.py", base_class=BaseModel, is_include_imported=False)
+ mock_register_file.assert_any_call(
+ file_path="/fake/file1.py", base_class=BaseModel, is_include_imported=False
+ )
+ mock_register_file.assert_any_call(
+ file_path="/fake/file2.py", base_class=BaseModel, is_include_imported=False
+ )