Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Update non-cursor agent rules
Louis Choqueldee7c0fbcc5a
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.mdindex 2d9aff5e6..fcbc54ea0 100644--- a/.github/copilot-instructions.md+++ b/.github/copilot-instructions.md@@ -1,4 +1,3 @@-Concatenation# Coding Standards & Best PracticesThis document outlines the core coding standards, best practices, and quality control procedures for the codebase.@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co## Type Hints1. **Always Use Type Hints**- - Every function parameter must be typed- - Every function return must be typed- - Use type hints for all variables where type is not obvious- - Use types with Uppercase first letter (Dict[], List[], etc.)++ - Every function parameter must be typed+ - Every function return must be typed+ - Use type hints for all variables where type is not obvious+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]+ - Use type hints for all fields+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`+ - Keep models focused and single-purpose2. **StrEnum**- Import from `pipelex.types`:@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality cofrom pipelex.types import StrEnum```-## BaseModel Standards--- Respect Pydantic v2 standards-- Keep models focused and single-purpose-- Use descriptive field names-- Use type hints for all fields-- Document complex validations-- Use Optional[] for nullable fields-- Use Field(default_factory=...) for mutable defaults+3. **Self type**+ - Import from `pipelex.types`:+ ```python+ from pipelex.types import Self+ ```## Factory Pattern-- Use Factory Pattern for object creation when dealing with multiple implementations+ - Use Factory Pattern for object creation when dealing with multiple implementations+ - Our factory methods are named `make_from_...` and such++## Error Handling++ - Always catch exceptions at the place where you can add useful context to it.+ - Use try/except blocks with specific exceptions+ - Convert third-party exceptions to our custom ones+ - Never catch Exception, only catch specific exceptions+ - Always add `from exc` to the exception++ ```python+ try:+ from fal_client import AsyncClient as FalAsyncClient+ except ImportError as exc:+ raise MissingDependencyError(+ "fal-client", "fal",+ "The fal-client SDK is required to use FAL models."+ ) from exc+ ```## Documentation@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co"""```-## Error Handling--1. **Graceful Error Handling**- - Use try/except blocks with specific exceptions- - Convert third-party exceptions to custom ones- ```python- try:- from fal_client import AsyncClient as FalAsyncClient- except ImportError as exc:- raise MissingDependencyError(- "fal-client", "fal",- "The fal-client SDK is required to use FAL models."- ) from exc- ```-## Code Quality Checks### Linting and Type Checking@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.- **Pipelines**: `pipelex/libraries/pipelines/`- **Tests**: `tests/` directory- **Documentation**: `docs/` directory-# Pipeline Guide++---++# Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural langage, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)-# Pipeline Structure Guide-## Pipeline File Naming- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)- Files must be `.py` for structures@@ -183,11 +189,11 @@ definition = "....."```The pipes will all have at least this base structure.-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).So If you have this error:`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •-variable='['ocr_input']'``-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.+variable='['invoice']'``+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:<NEVER DO THIS>@@ -201,9 +207,9 @@ inputs = {- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:-# Structured Models Rules+## Structuring Models-## Model Location and Registration+### Model Location and Registration- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`- Models must inherit from `StructuredContent` or appropriate content type@@ -232,7 +238,6 @@ If you simply need to refine another native concept, construct it like this:[concept.Landscape]refines = "Image"```-Only create a Python structure class when you need to add specific fields:```python@@ -255,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent# Date fields should remove timezonedate_field: Optional[datetime] = None```-## Usage+### UsageStructures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.-## Best Practices for structures+### Best Practices for structures- Respect Pydantic v2 standards- Use type hints for all fields- Use `Field` declaration and write the description-## Pipe Controllers and Pipe Operator+## Pipe Controllers and Pipe OperatorsLook at the Pipes we have in order to adapt it. Pipes are organized in two categories:@@ -277,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ- `PipeSequence` - For creating a sequence of multiple steps- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory- `PipeParallel` - For parallelizing pipes- - `PipeBatch` - For running pipes in Batch over a ListContent2. **Operators** - For specific tasks:- `PipeLLM` - Generate Text and Objects (include Vision LLM)- - `PipeOcr` - OCR Pipe+ - `PipeOcr` - Extract text and images from an image or a PDF+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.- `PipeImgGen` - Generate Images- `PipeFunc` - For running classic python scripts-# PipeSequence Guide+## PipeSequence controller-## Purpose-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.-## Basic Structure+### Basic Structure```plx[pipe.your_sequence_name]type = "PipeSequence"@@ -304,13 +308,13 @@ steps = []```-## Key Components+### Key Components1. **Steps Array**: List of pipes to execute in sequence- `pipe`: Name of the pipe to execute- `result`: Name to assign to the pipe's output that will be in the working memory-## Using PipeBatch in Steps+### Using PipeBatch in StepsYou can use PipeBatch functionality within steps using `batch_over` and `batch_as`:@@ -331,13 +335,11 @@ steps = [The result of a batched step will be a `ListContent` containing the outputs from processing each item.-# PipeCondition Controller+## PipeCondition controllerThe PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.-## Usage in PLX Configuration--### Basic Usage with Direct Expression+### Basic usage```plx[pipe.conditional_operation]@@ -367,7 +369,7 @@ medium = "process_medium"large = "process_large"```-## Key Parameters+### Key Parameters- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)@@ -375,41 +377,15 @@ large = "process_large"1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger-# PipeBatch Controller--The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.--## Usage in PLX Configuration--```plx-[pipe.sequence_with_batch]-type = "PipeSequence"-definition = "A Sequence of pipes"-inputs = { input_data = "ConceptName" }-output = "OutputConceptName"-steps = [- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }-]-```--## Key Parameters--- `pipe`: The pipe operation to apply to each element in the batch-- `batch_over`: The name of the list in the context to iterate over-- `batch_as`: The name to use for the current element in the pipe's context-- `result`: Where to store the results of the batch operation--# PipeLLM Guide--## Purpose+## PipeLLM operatorPipeLLM is used to:1. Generate text or objects with LLMs2. Process images with Vision LLMs-## Basic Usage+### Basic Usage-### Simple Text Generation+Simple Text Generation:```plx[pipe.write_story]type = "PipeLLM"@@ -420,7 +396,7 @@ Write a short story about a programmer."""```-### Structured Data Extraction+Structured Data Extraction:```plx[pipe.extract_info]type = "PipeLLM"@@ -433,8 +409,7 @@ Extract person information from this text:"""```-### System Prompts-Add system-level instructions:+Supports system instructions:```plx[pipe.expert_analysis]type = "PipeLLM"@@ -445,46 +420,95 @@ prompt_template = "Analyze this data"```### Multiple Outputs-Generate multiple results:++Generate multiple outputs (fixed number):```plx[pipe.generate_ideas]type = "PipeLLM"definition = "Generate ideas"output = "Idea"nb_output = 3 # Generate exactly 3 ideas-# OR+```++Generate multiple outputs (variable number):+```plx+[pipe.generate_ideas]+type = "PipeLLM"+definition = "Generate ideas"+output = "Idea"multiple_output = true # Let the LLM decide how many to generate```-### Vision Tasks+### Vision+Process images with VLMs:```plx[pipe.analyze_image]type = "PipeLLM"definition = "Analyze image"-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }output = "ImageAnalysis"prompt_template = "Describe what you see in this image"```-# PipeOCR Guide+### Writing prompts for PipeLLM-## Purpose+**Insert stuff inside a tagged block**-Extract text and images from an image or a PDF+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.-## Basic Usage+Example template:+```plx+prompt_template = """+Match the expense with its corresponding invoice:++@expense-### Simple Text Generation+@invoices+"""+```+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.++DO NOT write things like "Here is the expense: @expense".+DO write simply "@expense" alone in an isolated line.++**Insert stuff inline**++If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.++Example template:+```plx+prompt_template = """+Your goal is to summarize everything related to $topic in the provided text:++@text++Please provide only the summary, with no additional text or explanations.+Your summary should not be longer than 2 sentences.+"""+```++In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.+Be sure to make the proper choice of prefix for each insertion.++DO NOT write "$topic" alone in an isolated line.+DO write things like "Write an essay about $topic" to include text into an actual sentence.+++## PipeOcr operator++The PipeOcr operator is used to extract text and images from an image or a PDF++### Simple Text Extraction```plx[pipe.extract_info]type = "PipeOcr"definition = "extract the information"-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.output = "Page"```-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.+Only one input is allowed and it must either be an `Image` or a `PDF`.The output concept `Page` is a native concept, with the structure `PageContent`:It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`@@ -501,60 +525,134 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.-This rule explains how to write prompt templates in PipeLLM definitions.+----## Insert stuff inside a tagged block+## Rules to choose LLM models used in PipeLLMs.-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.+### LLM Configuration System-Example template:-```plx-prompt_template = """-Match the expense with its corresponding invoice:+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:-@expense+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`+- **Routing**: `.pipelex/inference/routing_profiles.toml`-@invoices+### LLM Handles++An llm_handle can be either:+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:++```toml+[aliases]+base-claude = "claude-4-sonnet"+base-gpt = "gpt-5"+base-gemini = "gemini-2.5-flash"+base-mistral = "mistral-medium"+```++The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.++### Using an LLM Handle in a PipeLLM++Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:++```plx+[pipe.hello_world]+type = "PipeLLM"+definition = "Write text about Hello World."+output = "Text"+llm = { llm_handle = "gpt-5", temperature = 0.9 }+prompt_template = """+Write a haiku about Hello World."""```-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.-**DO NOT write things like "Here is the expense: @expense".**-**DO write simply "@expense" alone in an isolated line.**+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").-## Insert stuff inline+### LLM Presets-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.++Examples:+```toml+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```++The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:-Example template:```plx+[pipe.extract_invoice]+type = "PipeLLM"+definition = "Extract invoice information from an invoice text transcript"+inputs = { invoice_text = "InvoiceText" }+output = "Invoice"+llm = "llm_to_extract_invoice"prompt_template = """-Your goal is to summarize everything related to $topic in the provided text:+Extract invoice information from this invoice:-@text+The category of this invoice is: $invoice_details.category.-Please provide only the summary, with no additional text or explanations.-Your summary should not be longer than 2 sentences.+@invoice_text"""```-Here, $topic will be inserted inline, whereas @text will be a a delimited block.-Be sure to make the proper choice of prefix for each insertion.+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.-**DO NOT write "$topic" alone in an isolated line.**-**DO write things like "Write an essay about $topic" included in an actual sentence.**+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.-# Example to execute a pipeline+---++ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+Then, create an example file to run the pipeline in the `examples` folder.+But don't write documentation unless asked explicitly to.++---++# Guide to write an example to execute a pipeline++## Example to execute a pipeline with text output```pythonimport asynciofrom pipelex import pretty_print-from pipelex.hub import get_pipeline_tracker, get_report_delegatefrom pipelex.pipelex import Pipelexfrom pipelex.pipeline.execute import execute_pipeline-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart++async def hello_world() -> str:+ """+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.+ """+ # Run the pipe+ pipe_output = await execute_pipeline(+ pipe_code="hello_world",+ )++ return pipe_output.main_stuff_as_str+++# start Pipelex+Pipelex.make()+# run sample using asyncio+output_text = asyncio.run(hello_world())+pretty_print(output_text, title="Your first Pipelex output")+```++## Example to execute a pipeline with structured output++```python+import asyncio++from pipelex import pretty_print+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execute import execute_pipeline++from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChartSAMPLE_NAME = "extract_gantt"IMAGE_URL = "assets/gantt/gantt_tree_house.png"@@ -579,22 +677,23 @@ async def extract_gantt(image_url: str) -> GanttChart:Pipelex.make()# run sample using asyncio-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))--# Display cost report (tokens used and cost)-get_report_delegate().generate_report()-# output results+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))pretty_print(gantt_chart, title="Gantt Chart")-get_pipeline_tracker().output_flowchart()```-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:+## Setting up the input memory++### Explanation of input memory++The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | strImplicitMemory = Dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.+### Different ways to set up the input memory+So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:```python@@ -613,7 +712,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wapipe_output = await execute_pipeline(pipe_code="power_extractor_dpe",input_memory={- "ocr_input": PDFContent(url=pdf_url),+ "document": PDFContent(url=pdf_url),},)@@ -654,92 +753,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa)```-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.-Then, create an example file to run the pipeline in the `examples` folder.-But don't write documentation unless asked explicitly to.+## Using the outputs of a pipeline-# Rules to choose LLM models used in PipeLLMs.+All pipe executions return a `PipeOutput` object.+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:-## LLM Configuration System+```python-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:+class PipeOutput(BaseModel):+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`-- **Routing**: `.pipelex/inference/routing_profiles.toml`+ @property+ def main_stuff(self) -> Stuff:+ ...-## LLM Handles+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:+ ...-An llm_handle can be either:-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:+ ...-```toml-[aliases]-best-claude = "claude-4.1-opus"-best-gemini = "gemini-2.5-pro"-best-mistral = "mistral-large"-base-gpt = "gpt-5"-```+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:+ ...-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.+ @property+ def main_stuff_as_text(self) -> TextContent:+ ...-## Using an LLM Handle in a PipeLLM+ @property+ def main_stuff_as_str(self) -> str:+ ...-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:+ @property+ def main_stuff_as_image(self) -> ImageContent:+ ...-```plx-[pipe.hello_world]-type = "PipeLLM"-definition = "Write text about Hello World."-output = "Text"-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }-prompt_template = """-Write a haiku about Hello World.-"""+ @property+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:+ ...++ @property+ def main_stuff_as_number(self) -> NumberContent:+ ...++ @property+ def main_stuff_as_html(self) -> HtmlContent:+ ...++ @property+ def main_stuff_as_mermaid(self) -> MermaidContent:+ ...```-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").+As you can see, you can extarct any variable from the output working memory.-## LLM Presets+### Getting the main stuff as a specific type-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.+Simple text as a string:-Examples:-```toml-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```python+result = pipe_output.main_stuff_as_str```+Structured object (BaseModel):-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:+```python+result = pipe_output.main_stuff_as(content_type=GanttChart)+```-```plx-[pipe.extract_invoice]-type = "PipeLLM"-definition = "Extract invoice information from an invoice text transcript"-inputs = { invoice_text = "InvoiceText" }-output = "Invoice"-llm = "llm_to_extract_invoice"-prompt_template = """-Extract invoice information from this invoice:+If it's a list, you can get a `ListContent` of the specific type.-The category of this invoice is: $invoice_details.category.+```python+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)+```-@invoice_text-"""+or if you want, you can get the actual items as a regular python list:++```python+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)```-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.+---+# Writing unit tests-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.+## Unit test generalities-These rules apply when writing unit tests.-- Always use pytest+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.-## Test file structure+### Test file structure- Name test files with `test_` prefix- Use descriptive names that match the functionality being tested@@ -753,7 +856,7 @@ These rules apply when writing unit tests.- Always put test inside Test classes.- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`-## Markers+### MarkersApply the appropriate markers:- "llm: uses an LLM to generate text or objects"@@ -763,11 +866,11 @@ Apply the appropriate markers:Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.-## Tips+### Important rules-- Never use the unittest.mock. Use pytest-mock+- Never use the unittest.mock. Use pytest-mock.-## Test Class Structure+### Test Class StructureAlways group the tests of a module into a test class:@@ -794,9 +897,9 @@ class TestFooBar:Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.-# Pipe tests+## Writing integration test to test pipes-## Required imports for pipe tests+### Required imports for pipe tests```pythonimport pytest@@ -815,7 +918,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParamsfrom pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol```-## Pipe test implementation steps+### Pipe test implementation steps1. Create Stuff from blueprint:@@ -845,14 +948,7 @@ pipe_output = await pipe_router.run_pipe()```-4. Log output and generate report:--```python-pretty_print(pipe_output, title=f"Pipe output")-get_report_delegate().generate_report()-```--5. Basic assertions:+4. Basic assertions:```pythonassert pipe_output is not None@@ -860,7 +956,7 @@ assert pipe_output.working_memory is not Noneassert pipe_output.main_stuff is not None```-## Test Data Organization+### Test Data Organization- If it's not already there, create a `test_data.py` file in the test directory- Define test cases using `StuffBlueprint`:@@ -897,28 +993,3 @@ Also note that we provide a topic for the test case, which is purely for conveni- Include docstrings explaining test purpose- Log outputs for debugging- Generate reports for cost tracking--# Test-Driven Development Guide--This document outlines our test-driven development (TDD) process and the tools available for testing.--## TDD Cycle--1. **Write a Test First**-[pytest.mdc](pytest.mdc)--2. **Write the Code**- - Implement the minimum amount of code needed to pass the test- - Follow the project's coding standards- - Keep it simple - don't write more than needed--3. **Run Linting and Type Checking**-[coding_standards.mdc](coding_standards.mdc)--4. **Refactor if needed**-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)--5. **Validate tests**--Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.-diff --git a/AGENTS.md b/AGENTS.mdindex c3d2602db..fcbc54ea0 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -1,4 +1,3 @@-Concatenation# Coding Standards & Best PracticesThis document outlines the core coding standards, best practices, and quality control procedures for the codebase.@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co## Type Hints1. **Always Use Type Hints**- - Every function parameter must be typed- - Every function return must be typed- - Use type hints for all variables where type is not obvious- - Use types with Uppercase first letter (Dict[], List[], etc.)++ - Every function parameter must be typed+ - Every function return must be typed+ - Use type hints for all variables where type is not obvious+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]+ - Use type hints for all fields+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`+ - Keep models focused and single-purpose2. **StrEnum**- Import from `pipelex.types`:@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality cofrom pipelex.types import StrEnum```-## BaseModel Standards--- Respect Pydantic v2 standards-- Keep models focused and single-purpose-- Use descriptive field names-- Use type hints for all fields-- Document complex validations-- Use Optional[] for nullable fields-- Use Field(default_factory=...) for mutable defaults+3. **Self type**+ - Import from `pipelex.types`:+ ```python+ from pipelex.types import Self+ ```## Factory Pattern-- Use Factory Pattern for object creation when dealing with multiple implementations+ - Use Factory Pattern for object creation when dealing with multiple implementations+ - Our factory methods are named `make_from_...` and such++## Error Handling++ - Always catch exceptions at the place where you can add useful context to it.+ - Use try/except blocks with specific exceptions+ - Convert third-party exceptions to our custom ones+ - Never catch Exception, only catch specific exceptions+ - Always add `from exc` to the exception++ ```python+ try:+ from fal_client import AsyncClient as FalAsyncClient+ except ImportError as exc:+ raise MissingDependencyError(+ "fal-client", "fal",+ "The fal-client SDK is required to use FAL models."+ ) from exc+ ```## Documentation@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co"""```-## Error Handling--1. **Graceful Error Handling**- - Use try/except blocks with specific exceptions- - Convert third-party exceptions to custom ones- ```python- try:- from fal_client import AsyncClient as FalAsyncClient- except ImportError as exc:- raise MissingDependencyError(- "fal-client", "fal",- "The fal-client SDK is required to use FAL models."- ) from exc- ```-## Code Quality Checks### Linting and Type Checking@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.- **Pipelines**: `pipelex/libraries/pipelines/`- **Tests**: `tests/` directory- **Documentation**: `docs/` directory-# Pipeline Guide++---++# Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural langage, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)-# Pipeline Structure Guide-## Pipeline File Naming- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)- Files must be `.py` for structures@@ -183,11 +189,11 @@ definition = "....."```The pipes will all have at least this base structure.-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).So If you have this error:`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •-variable='['ocr_input']'``-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.+variable='['invoice']'``+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:<NEVER DO THIS>@@ -201,9 +207,9 @@ inputs = {- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:-# Structured Models Rules+## Structuring Models-## Model Location and Registration+### Model Location and Registration- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`- Models must inherit from `StructuredContent` or appropriate content type@@ -254,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent# Date fields should remove timezonedate_field: Optional[datetime] = None```-## Usage+### UsageStructures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.-## Best Practices for structures+### Best Practices for structures- Respect Pydantic v2 standards- Use type hints for all fields- Use `Field` declaration and write the description-## Pipe Controllers and Pipe Operator+## Pipe Controllers and Pipe OperatorsLook at the Pipes we have in order to adapt it. Pipes are organized in two categories:@@ -276,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ- `PipeSequence` - For creating a sequence of multiple steps- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory- `PipeParallel` - For parallelizing pipes- - `PipeBatch` - For running pipes in Batch over a ListContent2. **Operators** - For specific tasks:- `PipeLLM` - Generate Text and Objects (include Vision LLM)- - `PipeOcr` - OCR Pipe+ - `PipeOcr` - Extract text and images from an image or a PDF+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.- `PipeImgGen` - Generate Images- `PipeFunc` - For running classic python scripts-# PipeSequence Guide+## PipeSequence controller-## Purpose-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.-## Basic Structure+### Basic Structure```plx[pipe.your_sequence_name]type = "PipeSequence"@@ -303,13 +308,13 @@ steps = []```-## Key Components+### Key Components1. **Steps Array**: List of pipes to execute in sequence- `pipe`: Name of the pipe to execute- `result`: Name to assign to the pipe's output that will be in the working memory-## Using PipeBatch in Steps+### Using PipeBatch in StepsYou can use PipeBatch functionality within steps using `batch_over` and `batch_as`:@@ -330,13 +335,11 @@ steps = [The result of a batched step will be a `ListContent` containing the outputs from processing each item.-# PipeCondition Controller+## PipeCondition controllerThe PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.-## Usage in PLX Configuration--### Basic Usage with Direct Expression+### Basic usage```plx[pipe.conditional_operation]@@ -366,7 +369,7 @@ medium = "process_medium"large = "process_large"```-## Key Parameters+### Key Parameters- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)@@ -374,41 +377,15 @@ large = "process_large"1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger-# PipeBatch Controller--The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.--## Usage in PLX Configuration--```plx-[pipe.sequence_with_batch]-type = "PipeSequence"-definition = "A Sequence of pipes"-inputs = { input_data = "ConceptName" }-output = "OutputConceptName"-steps = [- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }-]-```--## Key Parameters--- `pipe`: The pipe operation to apply to each element in the batch-- `batch_over`: The name of the list in the context to iterate over-- `batch_as`: The name to use for the current element in the pipe's context-- `result`: Where to store the results of the batch operation--# PipeLLM Guide--## Purpose+## PipeLLM operatorPipeLLM is used to:1. Generate text or objects with LLMs2. Process images with Vision LLMs-## Basic Usage+### Basic Usage-### Simple Text Generation+Simple Text Generation:```plx[pipe.write_story]type = "PipeLLM"@@ -419,7 +396,7 @@ Write a short story about a programmer."""```-### Structured Data Extraction+Structured Data Extraction:```plx[pipe.extract_info]type = "PipeLLM"@@ -432,8 +409,7 @@ Extract person information from this text:"""```-### System Prompts-Add system-level instructions:+Supports system instructions:```plx[pipe.expert_analysis]type = "PipeLLM"@@ -444,46 +420,95 @@ prompt_template = "Analyze this data"```### Multiple Outputs-Generate multiple results:++Generate multiple outputs (fixed number):```plx[pipe.generate_ideas]type = "PipeLLM"definition = "Generate ideas"output = "Idea"nb_output = 3 # Generate exactly 3 ideas-# OR+```++Generate multiple outputs (variable number):+```plx+[pipe.generate_ideas]+type = "PipeLLM"+definition = "Generate ideas"+output = "Idea"multiple_output = true # Let the LLM decide how many to generate```-### Vision Tasks+### Vision+Process images with VLMs:```plx[pipe.analyze_image]type = "PipeLLM"definition = "Analyze image"-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }output = "ImageAnalysis"prompt_template = "Describe what you see in this image"```-# PipeOCR Guide+### Writing prompts for PipeLLM-## Purpose+**Insert stuff inside a tagged block**-Extract text and images from an image or a PDF+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.-## Basic Usage+Example template:+```plx+prompt_template = """+Match the expense with its corresponding invoice:-### Simple Text Generation+@expense++@invoices+"""+```+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.++DO NOT write things like "Here is the expense: @expense".+DO write simply "@expense" alone in an isolated line.++**Insert stuff inline**++If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.++Example template:+```plx+prompt_template = """+Your goal is to summarize everything related to $topic in the provided text:++@text++Please provide only the summary, with no additional text or explanations.+Your summary should not be longer than 2 sentences.+"""+```++In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.+Be sure to make the proper choice of prefix for each insertion.++DO NOT write "$topic" alone in an isolated line.+DO write things like "Write an essay about $topic" to include text into an actual sentence.+++## PipeOcr operator++The PipeOcr operator is used to extract text and images from an image or a PDF++### Simple Text Extraction```plx[pipe.extract_info]type = "PipeOcr"definition = "extract the information"-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.output = "Page"```-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.+Only one input is allowed and it must either be an `Image` or a `PDF`.The output concept `Page` is a native concept, with the structure `PageContent`:It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`@@ -500,60 +525,134 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.-This rule explains how to write prompt templates in PipeLLM definitions.+----## Insert stuff inside a tagged block+## Rules to choose LLM models used in PipeLLMs.-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.+### LLM Configuration System-Example template:-```plx-prompt_template = """-Match the expense with its corresponding invoice:+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:-@expense+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`+- **Routing**: `.pipelex/inference/routing_profiles.toml`-@invoices+### LLM Handles++An llm_handle can be either:+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:++```toml+[aliases]+base-claude = "claude-4-sonnet"+base-gpt = "gpt-5"+base-gemini = "gemini-2.5-flash"+base-mistral = "mistral-medium"+```++The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.++### Using an LLM Handle in a PipeLLM++Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:++```plx+[pipe.hello_world]+type = "PipeLLM"+definition = "Write text about Hello World."+output = "Text"+llm = { llm_handle = "gpt-5", temperature = 0.9 }+prompt_template = """+Write a haiku about Hello World."""```-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.-**DO NOT write things like "Here is the expense: @expense".**-**DO write simply "@expense" alone in an isolated line.**+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").-## Insert stuff inline+### LLM Presets-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.++Examples:+```toml+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```++The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:-Example template:```plx+[pipe.extract_invoice]+type = "PipeLLM"+definition = "Extract invoice information from an invoice text transcript"+inputs = { invoice_text = "InvoiceText" }+output = "Invoice"+llm = "llm_to_extract_invoice"prompt_template = """-Your goal is to summarize everything related to $topic in the provided text:+Extract invoice information from this invoice:-@text+The category of this invoice is: $invoice_details.category.-Please provide only the summary, with no additional text or explanations.-Your summary should not be longer than 2 sentences.+@invoice_text"""```-Here, $topic will be inserted inline, whereas @text will be a a delimited block.-Be sure to make the proper choice of prefix for each insertion.+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.++You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.++---++ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+Then, create an example file to run the pipeline in the `examples` folder.+But don't write documentation unless asked explicitly to.++---++# Guide to write an example to execute a pipeline-**DO NOT write "$topic" alone in an isolated line.**-**DO write things like "Write an essay about $topic" included in an actual sentence.**+## Example to execute a pipeline with text output-# Example to execute a pipeline+```python+import asyncio++from pipelex import pretty_print+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execute import execute_pipeline+++async def hello_world() -> str:+ """+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.+ """+ # Run the pipe+ pipe_output = await execute_pipeline(+ pipe_code="hello_world",+ )++ return pipe_output.main_stuff_as_str+++# start Pipelex+Pipelex.make()+# run sample using asyncio+output_text = asyncio.run(hello_world())+pretty_print(output_text, title="Your first Pipelex output")+```++## Example to execute a pipeline with structured output```pythonimport asynciofrom pipelex import pretty_print-from pipelex.hub import get_pipeline_tracker, get_report_delegatefrom pipelex.pipelex import Pipelexfrom pipelex.pipeline.execute import execute_pipeline-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart+from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChartSAMPLE_NAME = "extract_gantt"IMAGE_URL = "assets/gantt/gantt_tree_house.png"@@ -578,22 +677,23 @@ async def extract_gantt(image_url: str) -> GanttChart:Pipelex.make()# run sample using asyncio-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))--# Display cost report (tokens used and cost)-get_report_delegate().generate_report()-# output results+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))pretty_print(gantt_chart, title="Gantt Chart")-get_pipeline_tracker().output_flowchart()```-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:+## Setting up the input memory++### Explanation of input memory++The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | strImplicitMemory = Dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.+### Different ways to set up the input memory+So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:```python@@ -612,7 +712,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wapipe_output = await execute_pipeline(pipe_code="power_extractor_dpe",input_memory={- "ocr_input": PDFContent(url=pdf_url),+ "document": PDFContent(url=pdf_url),},)@@ -653,92 +753,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa)```-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.-Then, create an example file to run the pipeline in the `examples` folder.-But don't write documentation unless asked explicitly to.+## Using the outputs of a pipeline-# Rules to choose LLM models used in PipeLLMs.+All pipe executions return a `PipeOutput` object.+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:-## LLM Configuration System+```python-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:+class PipeOutput(BaseModel):+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`-- **Routing**: `.pipelex/inference/routing_profiles.toml`+ @property+ def main_stuff(self) -> Stuff:+ ...-## LLM Handles+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:+ ...-An llm_handle can be either:-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:+ ...-```toml-[aliases]-best-claude = "claude-4.1-opus"-best-gemini = "gemini-2.5-pro"-best-mistral = "mistral-large"-base-gpt = "gpt-5"-```+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:+ ...-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.+ @property+ def main_stuff_as_text(self) -> TextContent:+ ...-## Using an LLM Handle in a PipeLLM+ @property+ def main_stuff_as_str(self) -> str:+ ...-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:+ @property+ def main_stuff_as_image(self) -> ImageContent:+ ...-```plx-[pipe.hello_world]-type = "PipeLLM"-definition = "Write text about Hello World."-output = "Text"-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }-prompt_template = """-Write a haiku about Hello World.-"""+ @property+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:+ ...++ @property+ def main_stuff_as_number(self) -> NumberContent:+ ...++ @property+ def main_stuff_as_html(self) -> HtmlContent:+ ...++ @property+ def main_stuff_as_mermaid(self) -> MermaidContent:+ ...```-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").+As you can see, you can extarct any variable from the output working memory.-## LLM Presets+### Getting the main stuff as a specific type-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.+Simple text as a string:-Examples:-```toml-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```python+result = pipe_output.main_stuff_as_str```+Structured object (BaseModel):-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:+```python+result = pipe_output.main_stuff_as(content_type=GanttChart)+```-```plx-[pipe.extract_invoice]-type = "PipeLLM"-definition = "Extract invoice information from an invoice text transcript"-inputs = { invoice_text = "InvoiceText" }-output = "Invoice"-llm = "llm_to_extract_invoice"-prompt_template = """-Extract invoice information from this invoice:+If it's a list, you can get a `ListContent` of the specific type.-The category of this invoice is: $invoice_details.category.+```python+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)+```-@invoice_text-"""+or if you want, you can get the actual items as a regular python list:++```python+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)```-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.+---+# Writing unit tests-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.+## Unit test generalities-These rules apply when writing unit tests.-- Always use pytest+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.-## Test file structure+### Test file structure- Name test files with `test_` prefix- Use descriptive names that match the functionality being tested@@ -752,7 +856,7 @@ These rules apply when writing unit tests.- Always put test inside Test classes.- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`-## Markers+### MarkersApply the appropriate markers:- "llm: uses an LLM to generate text or objects"@@ -762,11 +866,11 @@ Apply the appropriate markers:Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.-## Tips+### Important rules-- Never use the unittest.mock. Use pytest-mock+- Never use the unittest.mock. Use pytest-mock.-## Test Class Structure+### Test Class StructureAlways group the tests of a module into a test class:@@ -793,9 +897,9 @@ class TestFooBar:Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.-# Pipe tests+## Writing integration test to test pipes-## Required imports for pipe tests+### Required imports for pipe tests```pythonimport pytest@@ -814,7 +918,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParamsfrom pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol```-## Pipe test implementation steps+### Pipe test implementation steps1. Create Stuff from blueprint:@@ -844,14 +948,7 @@ pipe_output = await pipe_router.run_pipe()```-4. Log output and generate report:--```python-pretty_print(pipe_output, title=f"Pipe output")-get_report_delegate().generate_report()-```--5. Basic assertions:+4. Basic assertions:```pythonassert pipe_output is not None@@ -859,7 +956,7 @@ assert pipe_output.working_memory is not Noneassert pipe_output.main_stuff is not None```-## Test Data Organization+### Test Data Organization- If it's not already there, create a `test_data.py` file in the test directory- Define test cases using `StuffBlueprint`:@@ -896,28 +993,3 @@ Also note that we provide a topic for the test case, which is purely for conveni- Include docstrings explaining test purpose- Log outputs for debugging- Generate reports for cost tracking--# Test-Driven Development Guide--This document outlines our test-driven development (TDD) process and the tools available for testing.--## TDD Cycle--1. **Write a Test First**-[pytest.mdc](pytest.mdc)--2. **Write the Code**- - Implement the minimum amount of code needed to pass the test- - Follow the project's coding standards- - Keep it simple - don't write more than needed--3. **Run Linting and Type Checking**-[coding_standards.mdc](coding_standards.mdc)--4. **Refactor if needed**-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)--5. **Validate tests**--Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.-diff --git a/CLAUDE.md b/CLAUDE.mdindex c04686c77..fcbc54ea0 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -1,4 +1,3 @@-Concatenation# Coding Standards & Best PracticesThis document outlines the core coding standards, best practices, and quality control procedures for the codebase.@@ -6,10 +5,16 @@ This document outlines the core coding standards, best practices, and quality co## Type Hints1. **Always Use Type Hints**- - Every function parameter must be typed- - Every function return must be typed- - Use type hints for all variables where type is not obvious- - Use types with Uppercase first letter (Dict[], List[], etc.)++ - Every function parameter must be typed+ - Every function return must be typed+ - Use type hints for all variables where type is not obvious+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]+ - Use type hints for all fields+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`+ - Keep models focused and single-purpose2. **StrEnum**- Import from `pipelex.types`:@@ -17,19 +22,34 @@ This document outlines the core coding standards, best practices, and quality cofrom pipelex.types import StrEnum```-## BaseModel Standards--- Respect Pydantic v2 standards-- Keep models focused and single-purpose-- Use descriptive field names-- Use type hints for all fields-- Document complex validations-- Use Optional[] for nullable fields-- Use Field(default_factory=...) for mutable defaults+3. **Self type**+ - Import from `pipelex.types`:+ ```python+ from pipelex.types import Self+ ```## Factory Pattern-- Use Factory Pattern for object creation when dealing with multiple implementations+ - Use Factory Pattern for object creation when dealing with multiple implementations+ - Our factory methods are named `make_from_...` and such++## Error Handling++ - Always catch exceptions at the place where you can add useful context to it.+ - Use try/except blocks with specific exceptions+ - Convert third-party exceptions to our custom ones+ - Never catch Exception, only catch specific exceptions+ - Always add `from exc` to the exception++ ```python+ try:+ from fal_client import AsyncClient as FalAsyncClient+ except ImportError as exc:+ raise MissingDependencyError(+ "fal-client", "fal",+ "The fal-client SDK is required to use FAL models."+ ) from exc+ ```## Documentation@@ -57,21 +77,6 @@ This document outlines the core coding standards, best practices, and quality co"""```-## Error Handling--1. **Graceful Error Handling**- - Use try/except blocks with specific exceptions- - Convert third-party exceptions to custom ones- ```python- try:- from fal_client import AsyncClient as FalAsyncClient- except ImportError as exc:- raise MissingDependencyError(- "fal-client", "fal",- "The fal-client SDK is required to use FAL models."- ) from exc- ```-## Code Quality Checks### Linting and Type Checking@@ -118,14 +123,15 @@ Always fix any issues reported by these tools before proceeding.- **Pipelines**: `pipelex/libraries/pipelines/`- **Tests**: `tests/` directory- **Documentation**: `docs/` directory-# Pipeline Guide++---++# Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural langage, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.- Please use POSIX standard for files. (enmpty lines, no trailing whitespaces, etc.)-# Pipeline Structure Guide-## Pipeline File Naming- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)- Files must be `.py` for structures@@ -183,11 +189,11 @@ definition = "....."```The pipes will all have at least this base structure.-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).So If you have this error:`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •-variable='['ocr_input']'``-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.+variable='['invoice']'``+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:<NEVER DO THIS>@@ -201,9 +207,9 @@ inputs = {- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:-# Structured Models Rules+## Structuring Models-## Model Location and Registration+### Model Location and Registration- Create models for structured generations related to "some_domain" in `pipelex_libraries/pipelines/<some_domain>.py`- Models must inherit from `StructuredContent` or appropriate content type@@ -227,7 +233,6 @@ If a concept only refines a native concept (like Text, Image, etc.) without addi[concept]Joke = "A humorous text that makes people laugh."```-If you simply need to refine another native concept, construct it like this:```plx[concept.Landscape]@@ -255,21 +260,21 @@ class YourModel(StructuredContent): # Always be a subclass of StructuredContent# Date fields should remove timezonedate_field: Optional[datetime] = None```-## Usage+### UsageStructures are meant to indicate what class to use for a particular Concept. In general they use the same name as the concept.Structure classes defined within `pipelex_libraries/pipelines/` are automatically loaded into the class_registry when setting up Pipelex, no need to do it manually.-## Best Practices for structures+### Best Practices for structures- Respect Pydantic v2 standards- Use type hints for all fields- Use `Field` declaration and write the description-## Pipe Controllers and Pipe Operator+## Pipe Controllers and Pipe OperatorsLook at the Pipes we have in order to adapt it. Pipes are organized in two categories:@@ -277,20 +282,19 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ- `PipeSequence` - For creating a sequence of multiple steps- `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory- `PipeParallel` - For parallelizing pipes- - `PipeBatch` - For running pipes in Batch over a ListContent2. **Operators** - For specific tasks:- `PipeLLM` - Generate Text and Objects (include Vision LLM)- - `PipeOcr` - OCR Pipe+ - `PipeOcr` - Extract text and images from an image or a PDF+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.- `PipeImgGen` - Generate Images- `PipeFunc` - For running classic python scripts-# PipeSequence Guide+## PipeSequence controller-## Purpose-PipeSequence executes multiple pipes in a defined order, where each step can use results from previous steps.+Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.-## Basic Structure+### Basic Structure```plx[pipe.your_sequence_name]type = "PipeSequence"@@ -304,13 +308,13 @@ steps = []```-## Key Components+### Key Components1. **Steps Array**: List of pipes to execute in sequence- `pipe`: Name of the pipe to execute- `result`: Name to assign to the pipe's output that will be in the working memory-## Using PipeBatch in Steps+### Using PipeBatch in StepsYou can use PipeBatch functionality within steps using `batch_over` and `batch_as`:@@ -331,13 +335,11 @@ steps = [The result of a batched step will be a `ListContent` containing the outputs from processing each item.-# PipeCondition Controller+## PipeCondition controllerThe PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.-## Usage in PLX Configuration--### Basic Usage with Direct Expression+### Basic usage```plx[pipe.conditional_operation]@@ -367,7 +369,7 @@ medium = "process_medium"large = "process_large"```-## Key Parameters+### Key Parameters- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)@@ -375,41 +377,15 @@ large = "process_large"1 - The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`.2 - The value on the right (`process_small`, `process_medium`, ..) is the name of the pipce to trigger-# PipeBatch Controller--The PipeBatch controller allows you to apply a pipe operation to each element in a list of inputs in parallele. It is created via a PipeSequence.--## Usage in PLX Configuration--```plx-[pipe.sequence_with_batch]-type = "PipeSequence"-definition = "A Sequence of pipes"-inputs = { input_data = "ConceptName" }-output = "OutputConceptName"-steps = [- { pipe = "pipe_to_apply", batch_over = "input_list", batch_as = "current_item", result = "batch_results" }-]-```--## Key Parameters--- `pipe`: The pipe operation to apply to each element in the batch-- `batch_over`: The name of the list in the context to iterate over-- `batch_as`: The name to use for the current element in the pipe's context-- `result`: Where to store the results of the batch operation--# PipeLLM Guide--## Purpose+## PipeLLM operatorPipeLLM is used to:1. Generate text or objects with LLMs2. Process images with Vision LLMs-## Basic Usage+### Basic Usage-### Simple Text Generation+Simple Text Generation:```plx[pipe.write_story]type = "PipeLLM"@@ -420,7 +396,7 @@ Write a short story about a programmer."""```-### Structured Data Extraction+Structured Data Extraction:```plx[pipe.extract_info]type = "PipeLLM"@@ -433,8 +409,7 @@ Extract person information from this text:"""```-### System Prompts-Add system-level instructions:+Supports system instructions:```plx[pipe.expert_analysis]type = "PipeLLM"@@ -445,46 +420,95 @@ prompt_template = "Analyze this data"```### Multiple Outputs-Generate multiple results:++Generate multiple outputs (fixed number):```plx[pipe.generate_ideas]type = "PipeLLM"definition = "Generate ideas"output = "Idea"nb_output = 3 # Generate exactly 3 ideas-# OR+```++Generate multiple outputs (variable number):+```plx+[pipe.generate_ideas]+type = "PipeLLM"+definition = "Generate ideas"+output = "Idea"multiple_output = true # Let the LLM decide how many to generate```-### Vision Tasks+### Vision+Process images with VLMs:```plx[pipe.analyze_image]type = "PipeLLM"definition = "Analyze image"-inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in a stuff, you can add something like `{ "page.image": "Image" }+inputs = { image = "Image" } # `image` is the name of the stuff that contains the Image. If its in an attribute within a stuff, you can add something like `{ "page.image": "Image" }output = "ImageAnalysis"prompt_template = "Describe what you see in this image"```-# PipeOCR Guide+### Writing prompts for PipeLLM-## Purpose+**Insert stuff inside a tagged block**-Extract text and images from an image or a PDF+If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.-## Basic Usage+Example template:+```plx+prompt_template = """+Match the expense with its corresponding invoice:++@expense-### Simple Text Generation+@invoices+"""+```+In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.++DO NOT write things like "Here is the expense: @expense".+DO write simply "@expense" alone in an isolated line.++**Insert stuff inline**++If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.++Example template:+```plx+prompt_template = """+Your goal is to summarize everything related to $topic in the provided text:++@text++Please provide only the summary, with no additional text or explanations.+Your summary should not be longer than 2 sentences.+"""+```++In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.+Be sure to make the proper choice of prefix for each insertion.++DO NOT write "$topic" alone in an isolated line.+DO write things like "Write an essay about $topic" to include text into an actual sentence.+++## PipeOcr operator++The PipeOcr operator is used to extract text and images from an image or a PDF++### Simple Text Extraction```plx[pipe.extract_info]type = "PipeOcr"definition = "extract the information"-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.output = "Page"```-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.+Only one input is allowed and it must either be an `Image` or a `PDF`.The output concept `Page` is a native concept, with the structure `PageContent`:It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`@@ -501,60 +525,134 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.-This rule explains how to write prompt templates in PipeLLM definitions.+----## Insert stuff inside a tagged block+## Rules to choose LLM models used in PipeLLMs.-If the inserted text is supposedly long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.+### LLM Configuration System-Example template:-```plx-prompt_template = """-Match the expense with its corresponding invoice:+In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:-@expense+- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`+- **Routing**: `.pipelex/inference/routing_profiles.toml`-@invoices+### LLM Handles++An llm_handle can be either:+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:++```toml+[aliases]+base-claude = "claude-4-sonnet"+base-gpt = "gpt-5"+base-gemini = "gemini-2.5-flash"+base-mistral = "mistral-medium"+```++The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.++### Using an LLM Handle in a PipeLLM++Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:++```plx+[pipe.hello_world]+type = "PipeLLM"+definition = "Write text about Hello World."+output = "Text"+llm = { llm_handle = "gpt-5", temperature = 0.9 }+prompt_template = """+Write a haiku about Hello World."""```-In this example, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doens't need to be explictly written in the prompt template.-**DO NOT write things like "Here is the expense: @expense".**-**DO write simply "@expense" alone in an isolated line.**+As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").-## Insert stuff inline+### LLM Presets-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.+Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.++Examples:+```toml+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```++The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:-Example template:```plx+[pipe.extract_invoice]+type = "PipeLLM"+definition = "Extract invoice information from an invoice text transcript"+inputs = { invoice_text = "InvoiceText" }+output = "Invoice"+llm = "llm_to_extract_invoice"prompt_template = """-Your goal is to summarize everything related to $topic in the provided text:+Extract invoice information from this invoice:-@text+The category of this invoice is: $invoice_details.category.-Please provide only the summary, with no additional text or explanations.-Your summary should not be longer than 2 sentences.+@invoice_text"""```-Here, $topic will be inserted inline, whereas @text will be a a delimited block.-Be sure to make the proper choice of prefix for each insertion.+The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.-**DO NOT write "$topic" alone in an isolated line.**-**DO write things like "Write an essay about $topic" included in an actual sentence.**+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.-# Example to execute a pipeline+---++ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+Then, create an example file to run the pipeline in the `examples` folder.+But don't write documentation unless asked explicitly to.++---++# Guide to write an example to execute a pipeline++## Example to execute a pipeline with text output```pythonimport asynciofrom pipelex import pretty_print-from pipelex.hub import get_pipeline_tracker, get_report_delegatefrom pipelex.pipelex import Pipelexfrom pipelex.pipeline.execute import execute_pipeline-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart++async def hello_world() -> str:+ """+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.+ """+ # Run the pipe+ pipe_output = await execute_pipeline(+ pipe_code="hello_world",+ )++ return pipe_output.main_stuff_as_str+++# start Pipelex+Pipelex.make()+# run sample using asyncio+output_text = asyncio.run(hello_world())+pretty_print(output_text, title="Your first Pipelex output")+```++## Example to execute a pipeline with structured output++```python+import asyncio++from pipelex import pretty_print+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execute import execute_pipeline++from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChartSAMPLE_NAME = "extract_gantt"IMAGE_URL = "assets/gantt/gantt_tree_house.png"@@ -579,22 +677,23 @@ async def extract_gantt(image_url: str) -> GanttChart:Pipelex.make()# run sample using asyncio-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))--# Display cost report (tokens used and cost)-get_report_delegate().generate_report()-# output results+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))pretty_print(gantt_chart, title="Gantt Chart")-get_pipeline_tracker().output_flowchart()```-The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:+## Setting up the input memory++### Explanation of input memory++The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | strImplicitMemory = Dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.+### Different ways to set up the input memory+So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:```python@@ -613,7 +712,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wapipe_output = await execute_pipeline(pipe_code="power_extractor_dpe",input_memory={- "ocr_input": PDFContent(url=pdf_url),+ "document": PDFContent(url=pdf_url),},)@@ -654,92 +753,96 @@ So here are a few concrete examples of calls to execute_pipeline with various wa)```-ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.-Then, create an example file to run the pipeline in the `examples` folder.-But don't write documentation unless asked explicitly to.+## Using the outputs of a pipeline-# Rules to choose LLM models used in PipeLLMs.+All pipe executions return a `PipeOutput` object.+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:-## LLM Configuration System+```python-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:+class PipeOutput(BaseModel):+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`-- **Routing**: `.pipelex/inference/routing_profiles.toml`+ @property+ def main_stuff(self) -> Stuff:+ ...-## LLM Handles+ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:+ ...-An llm_handle can be either:-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:+ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:+ ...-```toml-[aliases]-best-claude = "claude-4.1-opus"-best-gemini = "gemini-2.5-pro"-best-mistral = "mistral-large"-base-gpt = "gpt-5"-```+ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:+ ...-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.+ @property+ def main_stuff_as_text(self) -> TextContent:+ ...-## Using an LLM Handle in a PipeLLM+ @property+ def main_stuff_as_str(self) -> str:+ ...-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:+ @property+ def main_stuff_as_image(self) -> ImageContent:+ ...-```plx-[pipe.hello_world]-type = "PipeLLM"-definition = "Write text about Hello World."-output = "Text"-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }-prompt_template = """-Write a haiku about Hello World.-"""+ @property+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:+ ...++ @property+ def main_stuff_as_number(self) -> NumberContent:+ ...++ @property+ def main_stuff_as_html(self) -> HtmlContent:+ ...++ @property+ def main_stuff_as_mermaid(self) -> MermaidContent:+ ...```-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").+As you can see, you can extarct any variable from the output working memory.-## LLM Presets+### Getting the main stuff as a specific type-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.+Simple text as a string:-Examples:-```toml-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }-llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```python+result = pipe_output.main_stuff_as_str```+Structured object (BaseModel):-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:+```python+result = pipe_output.main_stuff_as(content_type=GanttChart)+```-```plx-[pipe.extract_invoice]-type = "PipeLLM"-definition = "Extract invoice information from an invoice text transcript"-inputs = { invoice_text = "InvoiceText" }-output = "Invoice"-llm = "llm_to_extract_invoice"-prompt_template = """-Extract invoice information from this invoice:+If it's a list, you can get a `ListContent` of the specific type.-The category of this invoice is: $invoice_details.category.+```python+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)+```-@invoice_text-"""+or if you want, you can get the actual items as a regular python list:++```python+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)```-The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.+---+# Writing unit tests-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.+## Unit test generalities-These rules apply when writing unit tests.-- Always use pytest+NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.-## Test file structure+### Test file structure- Name test files with `test_` prefix- Use descriptive names that match the functionality being tested@@ -753,7 +856,7 @@ These rules apply when writing unit tests.- Always put test inside Test classes.- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`-## Markers+### MarkersApply the appropriate markers:- "llm: uses an LLM to generate text or objects"@@ -763,11 +866,11 @@ Apply the appropriate markers:Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.-## Tips+### Important rules-- Never use the unittest.mock. Use pytest-mock+- Never use the unittest.mock. Use pytest-mock.-## Test Class Structure+### Test Class StructureAlways group the tests of a module into a test class:@@ -794,9 +897,9 @@ class TestFooBar:Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.-# Pipe tests+## Writing integration test to test pipes-## Required imports for pipe tests+### Required imports for pipe tests```pythonimport pytest@@ -815,7 +918,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParamsfrom pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol```-## Pipe test implementation steps+### Pipe test implementation steps1. Create Stuff from blueprint:@@ -845,14 +948,7 @@ pipe_output = await pipe_router.run_pipe()```-4. Log output and generate report:--```python-pretty_print(pipe_output, title=f"Pipe output")-get_report_delegate().generate_report()-```--5. Basic assertions:+4. Basic assertions:```pythonassert pipe_output is not None@@ -860,7 +956,7 @@ assert pipe_output.working_memory is not Noneassert pipe_output.main_stuff is not None```-## Test Data Organization+### Test Data Organization- If it's not already there, create a `test_data.py` file in the test directory- Define test cases using `StuffBlueprint`:@@ -897,28 +993,3 @@ Also note that we provide a topic for the test case, which is purely for conveni- Include docstrings explaining test purpose- Log outputs for debugging- Generate reports for cost tracking--# Test-Driven Development Guide--This document outlines our test-driven development (TDD) process and the tools available for testing.--## TDD Cycle--1. **Write a Test First**-[pytest.mdc](pytest.mdc)--2. **Write the Code**- - Implement the minimum amount of code needed to pass the test- - Follow the project's coding standards- - Keep it simple - don't write more than needed--3. **Run Linting and Type Checking**-[coding_standards.mdc](coding_standards.mdc)--4. **Refactor if needed**-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)--5. **Validate tests**--Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.-