Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Feature/support codex (#467)

Louis Choquelfa35a57afd27
diff --git a/.blackboxrules b/.blackboxrules
index fa12db087..211d85f6e 100644
--- a/.blackboxrules
+++ b/.blackboxrules
@@ -1,1175 +1,329 @@
<!-- BEGIN_PIPELEX_RULES -->
-## Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-### Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-### Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-#### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-#### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-#### Pipe Definitions
-
-### Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-#### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-## Single item (default)
-inputs = { document = "Text" }
-
-## Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-## Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
+# Pipelex Coding Rules
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
+## Commands
-### Structuring Models
+### Linting
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-#### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
-
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
-
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-#### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-#### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-### Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-### PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-#### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-#### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-#### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-### PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-#### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-#### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
+ After making code changes, you must always lint using `make check`.
-### PipeLLM operator
-
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-#### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-#### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-#### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-#### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-### PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-#### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
+ Always fix any issues reported by these tools before proceeding.
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
+### Running Tests
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
-### PipeCompose operator
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
+### Running Tests with Prints
-#### Basic Usage
+ If anything went wrong, you can run the tests with prints to see the error:
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-## Report Summary
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
-Based on the analysis:
-$data
+### Running specific Tests
-Generated on: {{ current_date }}
-"""
-```
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
+### Running Last Failed Tests
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
+ To rerun only the tests that failed in the previous run, use:
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
-Hi $customer.first_name,
+---
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+### Prerequisites for running command lines: activate virtual environment
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+ Do this:
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
+ or do that:
-Looking forward to moving this forward together!
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
+ (adapt the above command to the OS and available virtual environment name)
-#### Key Parameters
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-#### Template Variables
+## Coding Standards & Best Practices for Python Code
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
+This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
-### PipeImgGen operator
+### Variables, loops and indexes
-The PipeImgGen operator is used to generate images using AI image generation models.
+ - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
+ - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
+ - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
+ - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.
-#### Basic Usage
+### Enums and tests
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
+ - When defining enums related to string values, always inherit from `StrEnum`
+ - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
+ - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
+### Imports
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
+#### **Imports at the top of the file**
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
+ - Import all necessary libraries at the top of the file
+ - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
-#### Key Parameters
+- **Logging and Pretty Printing**:
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
+ - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
+ ```python
+ from pipelex import log, pretty_print
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
+ log.info("Hello, world!")
+ ```
+ - Both have a title arg which is handy when logging/printing objects:
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
+ ```python
+ log.verbose("Hello, world!", title="Your first Pipelex log")
+ pretty_print(output_object, title="Your first Pipelex output")
+ ```
+ - Both handle formatting json using Rich, pretty_print makes it prettier.
-#### Input Requirements
+- **StrEnum and Self type**:
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
+ - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import Self, StrEnum
+ ```
-The input can be named anything but must contain the prompt text for image generation.
+### Typing
-### PipeFunc operator
+#### **Always Use Type Hints**
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use Field(default_factory=...) for mutable defaults
+ - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
-#### Basic Usage
+#### **BaseModel / Pydantic Standards**
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
+ - Use `BaseModel` and respect Pydantic v2 standards
+ - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
+ - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
+ ```python
+ from pydantic import BaseModel, Field
+ from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
+
+ class MyModel(BaseModel):
+ names: list[str] = Field(default_factory=list) # OK for strings
+ numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
+ items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
+ ```
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
+### Factory Pattern
-#### Key Parameters
+ - Use factory pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
+### Error Handling
-#### Function Requirements
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
+ - NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
-The Python function must:
+### Documentation
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
+1. **Docstring Format**
```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
+ def process_image(image_path: str, size: tuple[int, int]) -> bytes:
+ """Process and resize an image.
+
+ Args:
+ image_path: Path to the source image
+ size: Tuple of (width, height) for resizing
+
+ Returns:
+ Processed image as bytes
+ """
pass
```
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-#### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-#### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-### Rules to choose LLM models used in PipeLLMs.
-
-#### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-#### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-#### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-#### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
-
-## Guide to execute a pipeline and write example code
-
-### Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-## Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-## or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-### Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-## start Pipelex
-Pipelex.make()
-## run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-### Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
+2. **Class Documentation**
+ ```python
+ class ImageProcessor:
+ """Handles image processing operations.
+
+ Provides methods for resizing, converting, and optimizing images.
+ """
+ ```
-from my_project.gantt.gantt_struct import GanttChart
+## Writing tests
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
+### Unit test generalities
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
+#### Test file structure
+- Name test files with `test_` prefix
+- Place test files in the appropriate test category directory:
+ - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+ - `tests/integration/` - for integration tests that test component interactions
+ - `tests/e2e/` - for end-to-end tests that test complete workflows
+ - `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
-## start Pipelex
-Pipelex.make()
+#### Markers
-## run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
+Apply the appropriate markers:
+- "llm: uses an LLM to generate text or objects"
+- "img_gen: uses an image generation AI"
+- "extract: uses text/image extraction from documents"
+- "inference: uses either an LLM or an image generation AI"
+- "gha_disabled: will not be able to run properly on GitHub Actions"
-### Setting up the input memory
+Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-#### Explanation of input memory
+#### Important rules
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+- Never use the unittest.mock. Use pytest-mock.
-#### Different ways to set up the input memory
+#### Test Class Structure
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
+- Always group the tests of a module into a test class:
```python
-## Here we have a single input and it's a Text.
-## If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-## Here we have a single input and it's a PDF.
-## Because PDFContent is a native concept, we can use it directly as a value,
-## the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-## Here we have a single input and it's an Image.
-## Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-## Here we have a single input, it's an image but
-## its actually a more specific concept gantt.GanttImage which refines Image,
-## so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
[email protected](loop_scope="class")
+class TestFooBar:
+ @pytest.mark.parametrize(
+ "topic, test_case_blueprint",
+ [
+ TestCases.CASE_1,
+ TestCases.CASE_2,
+ ],
)
-
-## Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
- )
-```
-
-### Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
-
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
-
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
-
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
-
- @property
- def main_stuff_as_str(self) -> str:
- ...
-
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
-
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
-
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
-
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
-
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
+ async def test_pipe_processing(
+ self,
+ request: FixtureRequest,
+ topic: str,
+ test_case_blueprint: StuffBlueprint,
+ ):
+ # Test implementation
```
-As you can see, you can extract any variable from the output working memory.
-
-#### Getting the main stuff as a specific type
-
-Simple text as a string:
-
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
+- Never more than 1 class per test module.
+- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
+- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
+#### Test Data Organization
-If it's a list, you can get a `ListContent` of the specific type.
+- If it's not already there, create a `test_data.py` file in the proper test directory
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
+### Best Practices for Testing
-or if you want, you can get the actual items as a regular python list:
+- Use strong asserts: test value, not just type and presence.
+- Use parametrize for multiple test cases
+- Test both success and failure cases
+- Verify working memory state
+- Check output structure and content
+- Use meaningful test case names
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
+- Log outputs for debugging
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
+## Writing Docs
----
+We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
-## Rules to choose LLM models used in PipeLLMs.
+### MkDocs Markdown Requirements
-### LLM Configuration System
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+## Test-Driven Development Guide
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+This document outlines our test-driven development (TDD) process and the tools available for testing.
-### LLM Handles
+### TDD Cycle
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+1. **Write a Test First**
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
+2. **Write the Code**
+ - Implement the minimum amount of code needed to pass the test
+ - Follow the project's coding standards
+ - Keep it simple - don't write more than needed
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+3. **Run Linting and Type Checking**
+4. **Validate tests**
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
<!-- END_PIPELEX_RULES -->
diff --git a/.cursor/rules/commands.mdc b/.cursor/rules/commands.mdc
index 6c017d195..279b9b820 100644
--- a/.cursor/rules/commands.mdc
+++ b/.cursor/rules/commands.mdc
@@ -4,5 +4,92 @@ description: Guidelines for running commands
---
# Commands
- - When you want to run commands such as `python`, `pytest` or any of our CLI such as `pipelex` or `cocode`, ALWAYS use the current obvious virtual env. If the installaton is standard, the venv is named `.venv` so always check that first.
-
\ No newline at end of file
+## Linting
+
+ After making code changes, you must always lint using `make check`.
+
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
+
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
+
+ Always fix any issues reported by these tools before proceeding.
+
+## Running Tests
+
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
+
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
+
+## Running Tests with Prints
+
+ If anything went wrong, you can run the tests with prints to see the error:
+
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
+
+## Running specific Tests
+
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
+
+## Running Last Failed Tests
+
+ To rerun only the tests that failed in the previous run, use:
+
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
+
+---
+
+## Prerequisites for running command lines: activate virtual environment
+
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
+
+ Do this:
+
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
+
+ or do that:
+
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
+
+ (adapt the above command to the OS and available virtual environment name)
+
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
+
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
+
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
diff --git a/.cursor/rules/docs.mdc b/.cursor/rules/docs.mdc
index 1a166508b..f81bc9c42 100644
--- a/.cursor/rules/docs.mdc
+++ b/.cursor/rules/docs.mdc
@@ -6,8 +6,6 @@ globs:
---
# Writing Docs
-Write docs and answer questions about writing docs.
-
We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
## MkDocs Markdown Requirements
diff --git a/.cursor/rules/llms.mdc b/.cursor/rules/llms.mdc
deleted file mode 100644
index 7622b337d..000000000
--- a/.cursor/rules/llms.mdc
+++ /dev/null
@@ -1,85 +0,0 @@
----
-alwaysApply: false
-description: LLM configuration and usage guidelines
-globs:
-- '*.plx'
-- '*.toml'
----
-# Rules to choose LLM models used in PipeLLMs.
-
-## LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-## LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-## Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-## LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
diff --git a/.cursor/rules/pytest_standards.mdc b/.cursor/rules/pytest_standards.mdc
index b1c2f11e2..2b1a92002 100644
--- a/.cursor/rules/pytest_standards.mdc
+++ b/.cursor/rules/pytest_standards.mdc
@@ -4,11 +4,11 @@ description: Guidelines for writing unit tests
globs:
- tests/**/*.py
---
-# Writing unit tests
+# Writing tests
## Unit test generalities
-NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
### Test file structure
@@ -19,9 +19,8 @@ NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
-- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `tests.pipelex.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
-- Always put test inside Test classes.
-- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
### Markers
@@ -63,45 +62,23 @@ class TestFooBar:
# Test implementation
```
-- Never more than 1 class per test file.
+- Never more than 1 class per test module.
- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
### Test Data Organization
- If it's not already there, create a `test_data.py` file in the proper test directory
-- Define test cases using `StuffBlueprint`:
-
-```python
-class TestCases:
- CASE_BLUEPRINT_1 = StuffBlueprint(
- name="test_case_1",
- concept_code="domain.ConceptName1",
- value="test_value"
- )
- CASE_BLUEPRINT_2 = StuffBlueprint(
- name="test_case_2",
- concept_code="domain.ConceptName2",
- value="test_value"
- )
-
- CASE_BLUEPRINTS: ClassVar[list[tuple[str, str]]] = [ # topic, blueprint"
- ("topic1", CASE_BLUEPRINT_1),
- ("topic2", CASE_BLUEPRINT_2),
- ]
-```
-
-Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
-Also note that we provide a topic for the test case, which is purely for convenience.
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
## Best Practices for Testing
-- Whenever possible, use strong asserts to test value, not just type and presence.
+- Use strong asserts: test value, not just type and presence.
- Use parametrize for multiple test cases
- Test both success and failure cases
- Verify working memory state
- Check output structure and content
- Use meaningful test case names
-- Include docstrings explaining test purpose but not on top of the file
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
- Log outputs for debugging
-- Generate reports for cost tracking
diff --git a/.cursor/rules/python_standards.mdc b/.cursor/rules/python_standards.mdc
index 2e2e0a9b2..980b3c7d6 100644
--- a/.cursor/rules/python_standards.mdc
+++ b/.cursor/rules/python_standards.mdc
@@ -20,6 +20,10 @@ This document outlines the core coding standards, best practices, and quality co
- When defining enums related to string values, always inherit from `StrEnum`
- Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
- As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
## Imports
@@ -82,17 +86,17 @@ This document outlines the core coding standards, best practices, and quality co
## Factory Pattern
- - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use factory pattern for object creation when dealing with multiple implementations
- Our factory methods are named `make_from_...` and such
## Error Handling
- Always catch exceptions at the place where you can add useful context to it.
- Use try/except blocks with specific exceptions
- - Convert third-party exceptions to our custom ones
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
- NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
- - NEVER raise generic exceptions like ValueError or TypeError, create new error classes and raise them instead
- - Always add `from exc` to the exception
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
```python
try:
@@ -102,8 +106,6 @@ This document outlines the core coding standards, best practices, and quality co
raise PipelexSetupError(msg) from exc
```
- **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
-
## Documentation
1. **Docstring Format**
@@ -129,48 +131,3 @@ This document outlines the core coding standards, best practices, and quality co
Provides methods for resizing, converting, and optimizing images.
"""
```
-
-## Code Quality Checks
-
-### Linting and Type Checking
-
-Before finalizing a task, run:
-```bash
-make fix-unused-imports
-make check
-```
-
-This runs multiple code quality tools:
-- Pyright: Static type checking
-- Ruff: Fast Python linter
-- Mypy: Static type checker
-
-Always fix any issues reported by these tools before proceeding.
-
-### Running Tests
-
-1. **Quick Test Run** (no LLM/image generation):
- ```bash
- make tp
- ```
- Runs tests with markers: `(dry_runnable or not (inference or llm or img_gen or extract)) and not (needs_output or pipelex_api)`
-
-2. **Specific Tests**:
- ```bash
- make tp TEST=TestClassName
- # or
- make tp TEST=test_function_name
- ```
- Note: Matches names starting with the provided string.
-
-**Important**: Never run `make ti`, `make test-inference`, `make te`, `make test-extract`, `make tg`, or `make test-img-gen` - these use costly inference.
-
-## Pipelines
-
-- Always validate pipelines after creation/edit with `make validate`.
- Iterate if there are errors.
-
-## Project Structure
-
-- **Tests**: `tests/` directory
-- **Documentation**: `docs/` directory
diff --git a/.cursor/rules/run_pipelex.mdc b/.cursor/rules/run_pipelex.mdc
deleted file mode 100644
index 78aed7449..000000000
--- a/.cursor/rules/run_pipelex.mdc
+++ /dev/null
@@ -1,246 +0,0 @@
----
-alwaysApply: false
-description: Guidelines for running Pipelex pipelines
-globs:
-- examples/**/*.py
----
-# Guide to execute a pipeline and write example code
-
-## Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-# Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-# or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-## Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-# start Pipelex
-Pipelex.make()
-# run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-## Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
-
-from my_project.gantt.gantt_struct import GanttChart
-
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
-
-
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
-
-
-# start Pipelex
-Pipelex.make()
-
-# run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
-
-## Setting up the input memory
-
-### Explanation of input memory
-
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
-
-### Different ways to set up the input memory
-
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
-
-```python
-# Here we have a single input and it's a Text.
-# If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-# Here we have a single input and it's a PDF.
-# Because PDFContent is a native concept, we can use it directly as a value,
-# the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-# Here we have a single input and it's an Image.
-# Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-# Here we have a single input, it's an image but
-# its actually a more specific concept gantt.GanttImage which refines Image,
-# so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
-
-# Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
- )
-```
-
-## Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
-
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
-
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
-
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
-
- @property
- def main_stuff_as_str(self) -> str:
- ...
-
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
-
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
-
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
-
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
-
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
-```
-
-As you can see, you can extract any variable from the output working memory.
-
-### Getting the main stuff as a specific type
-
-Simple text as a string:
-
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
-
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
-
-If it's a list, you can get a `ListContent` of the specific type.
-
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
-
-or if you want, you can get the actual items as a regular python list:
-
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
-
----
-
diff --git a/.cursor/rules/tdd.mdc b/.cursor/rules/tdd.mdc
index 4b4f058b5..0342df336 100644
--- a/.cursor/rules/tdd.mdc
+++ b/.cursor/rules/tdd.mdc
@@ -9,7 +9,6 @@ This document outlines our test-driven development (TDD) process and the tools a
## TDD Cycle
1. **Write a Test First**
-[pytest.mdc](pytest.mdc)
2. **Write the Code**
- Implement the minimum amount of code needed to pass the test
@@ -17,12 +16,8 @@ This document outlines our test-driven development (TDD) process and the tools a
- Keep it simple - don't write more than needed
3. **Run Linting and Type Checking**
-[coding_standards.mdc](coding_standards.mdc)
-4. **Refactor if needed**
-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)
+4. **Validate tests**
-5. **Validate tests**
-
-Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
diff --git a/.cursor/rules/write_pipelex.mdc b/.cursor/rules/write_pipelex.mdc
deleted file mode 100644
index b0695ea65..000000000
--- a/.cursor/rules/write_pipelex.mdc
+++ /dev/null
@@ -1,862 +0,0 @@
----
-alwaysApply: false
-description: Guidelines for writing Pipelex pipelines
-globs:
-- '**/*.plx'
-- '**/pipelines/**/*.py'
----
-# Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-## Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-## Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-### Pipe Definitions
-
-## Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-# Single item (default)
-inputs = { document = "Text" }
-
-# Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-# Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
-
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
-
-## Structuring Models
-
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
-
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
-
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-## Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-## PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-## PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
-
-## PipeLLM operator
-
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-## PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
-
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
-
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
-
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
-
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
-
-## PipeCompose operator
-
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
-
-### Basic Usage
-
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-# Report Summary
-
-Based on the analysis:
-$data
-
-Generated on: {{ current_date }}
-"""
-```
-
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
-
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
-
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
-
-Hi $customer.first_name,
-
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
-
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
-
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
-
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
-
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
-
-Looking forward to moving this forward together!
-
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
-
-### Key Parameters
-
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
-
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
-
-### Template Variables
-
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
-
-## PipeImgGen operator
-
-The PipeImgGen operator is used to generate images using AI image generation models.
-
-### Basic Usage
-
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
-
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
-
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
-
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
-
-### Key Parameters
-
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
-
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
-
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
-
-### Input Requirements
-
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
-
-The input can be named anything but must contain the prompt text for image generation.
-
-## PipeFunc operator
-
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
-
-### Basic Usage
-
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
-
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
-
-### Key Parameters
-
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
-
-### Function Requirements
-
-The Python function must:
-
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
- ```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
- pass
- ```
-
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-## Rules to choose LLM models used in PipeLLMs.
-
-### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index fa12db087..211d85f6e 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -1,1175 +1,329 @@
<!-- BEGIN_PIPELEX_RULES -->
-## Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-### Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-### Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-#### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-#### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-#### Pipe Definitions
-
-### Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-#### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-## Single item (default)
-inputs = { document = "Text" }
-
-## Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-## Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
+# Pipelex Coding Rules
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
+## Commands
-### Structuring Models
+### Linting
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-#### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
-
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
-
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-#### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-#### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-### Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-### PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-#### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-#### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-#### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-### PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-#### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-#### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
+ After making code changes, you must always lint using `make check`.
-### PipeLLM operator
-
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-#### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-#### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-#### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-#### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-### PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-#### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
+ Always fix any issues reported by these tools before proceeding.
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
+### Running Tests
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
-### PipeCompose operator
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
+### Running Tests with Prints
-#### Basic Usage
+ If anything went wrong, you can run the tests with prints to see the error:
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-## Report Summary
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
-Based on the analysis:
-$data
+### Running specific Tests
-Generated on: {{ current_date }}
-"""
-```
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
+### Running Last Failed Tests
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
+ To rerun only the tests that failed in the previous run, use:
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
-Hi $customer.first_name,
+---
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+### Prerequisites for running command lines: activate virtual environment
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+ Do this:
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
+ or do that:
-Looking forward to moving this forward together!
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
+ (adapt the above command to the OS and available virtual environment name)
-#### Key Parameters
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-#### Template Variables
+## Coding Standards & Best Practices for Python Code
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
+This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
-### PipeImgGen operator
+### Variables, loops and indexes
-The PipeImgGen operator is used to generate images using AI image generation models.
+ - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
+ - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
+ - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
+ - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.
-#### Basic Usage
+### Enums and tests
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
+ - When defining enums related to string values, always inherit from `StrEnum`
+ - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
+ - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
+### Imports
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
+#### **Imports at the top of the file**
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
+ - Import all necessary libraries at the top of the file
+ - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
-#### Key Parameters
+- **Logging and Pretty Printing**:
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
+ - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
+ ```python
+ from pipelex import log, pretty_print
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
+ log.info("Hello, world!")
+ ```
+ - Both have a title arg which is handy when logging/printing objects:
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
+ ```python
+ log.verbose("Hello, world!", title="Your first Pipelex log")
+ pretty_print(output_object, title="Your first Pipelex output")
+ ```
+ - Both handle formatting json using Rich, pretty_print makes it prettier.
-#### Input Requirements
+- **StrEnum and Self type**:
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
+ - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import Self, StrEnum
+ ```
-The input can be named anything but must contain the prompt text for image generation.
+### Typing
-### PipeFunc operator
+#### **Always Use Type Hints**
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use Field(default_factory=...) for mutable defaults
+ - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
-#### Basic Usage
+#### **BaseModel / Pydantic Standards**
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
+ - Use `BaseModel` and respect Pydantic v2 standards
+ - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
+ - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
+ ```python
+ from pydantic import BaseModel, Field
+ from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
+
+ class MyModel(BaseModel):
+ names: list[str] = Field(default_factory=list) # OK for strings
+ numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
+ items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
+ ```
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
+### Factory Pattern
-#### Key Parameters
+ - Use factory pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
+### Error Handling
-#### Function Requirements
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
+ - NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
-The Python function must:
+### Documentation
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
+1. **Docstring Format**
```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
+ def process_image(image_path: str, size: tuple[int, int]) -> bytes:
+ """Process and resize an image.
+
+ Args:
+ image_path: Path to the source image
+ size: Tuple of (width, height) for resizing
+
+ Returns:
+ Processed image as bytes
+ """
pass
```
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-#### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-#### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-### Rules to choose LLM models used in PipeLLMs.
-
-#### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-#### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-#### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-#### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
-
-## Guide to execute a pipeline and write example code
-
-### Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-## Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-## or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-### Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-## start Pipelex
-Pipelex.make()
-## run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-### Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
+2. **Class Documentation**
+ ```python
+ class ImageProcessor:
+ """Handles image processing operations.
+
+ Provides methods for resizing, converting, and optimizing images.
+ """
+ ```
-from my_project.gantt.gantt_struct import GanttChart
+## Writing tests
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
+### Unit test generalities
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
+#### Test file structure
+- Name test files with `test_` prefix
+- Place test files in the appropriate test category directory:
+ - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+ - `tests/integration/` - for integration tests that test component interactions
+ - `tests/e2e/` - for end-to-end tests that test complete workflows
+ - `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
-## start Pipelex
-Pipelex.make()
+#### Markers
-## run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
+Apply the appropriate markers:
+- "llm: uses an LLM to generate text or objects"
+- "img_gen: uses an image generation AI"
+- "extract: uses text/image extraction from documents"
+- "inference: uses either an LLM or an image generation AI"
+- "gha_disabled: will not be able to run properly on GitHub Actions"
-### Setting up the input memory
+Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-#### Explanation of input memory
+#### Important rules
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+- Never use the unittest.mock. Use pytest-mock.
-#### Different ways to set up the input memory
+#### Test Class Structure
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
+- Always group the tests of a module into a test class:
```python
-## Here we have a single input and it's a Text.
-## If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-## Here we have a single input and it's a PDF.
-## Because PDFContent is a native concept, we can use it directly as a value,
-## the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-## Here we have a single input and it's an Image.
-## Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-## Here we have a single input, it's an image but
-## its actually a more specific concept gantt.GanttImage which refines Image,
-## so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
[email protected](loop_scope="class")
+class TestFooBar:
+ @pytest.mark.parametrize(
+ "topic, test_case_blueprint",
+ [
+ TestCases.CASE_1,
+ TestCases.CASE_2,
+ ],
)
-
-## Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
- )
-```
-
-### Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
-
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
-
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
-
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
-
- @property
- def main_stuff_as_str(self) -> str:
- ...
-
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
-
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
-
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
-
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
-
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
+ async def test_pipe_processing(
+ self,
+ request: FixtureRequest,
+ topic: str,
+ test_case_blueprint: StuffBlueprint,
+ ):
+ # Test implementation
```
-As you can see, you can extract any variable from the output working memory.
-
-#### Getting the main stuff as a specific type
-
-Simple text as a string:
-
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
+- Never more than 1 class per test module.
+- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
+- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
+#### Test Data Organization
-If it's a list, you can get a `ListContent` of the specific type.
+- If it's not already there, create a `test_data.py` file in the proper test directory
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
+### Best Practices for Testing
-or if you want, you can get the actual items as a regular python list:
+- Use strong asserts: test value, not just type and presence.
+- Use parametrize for multiple test cases
+- Test both success and failure cases
+- Verify working memory state
+- Check output structure and content
+- Use meaningful test case names
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
+- Log outputs for debugging
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
+## Writing Docs
----
+We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
-## Rules to choose LLM models used in PipeLLMs.
+### MkDocs Markdown Requirements
-### LLM Configuration System
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+## Test-Driven Development Guide
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+This document outlines our test-driven development (TDD) process and the tools available for testing.
-### LLM Handles
+### TDD Cycle
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+1. **Write a Test First**
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
+2. **Write the Code**
+ - Implement the minimum amount of code needed to pass the test
+ - Follow the project's coding standards
+ - Keep it simple - don't write more than needed
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+3. **Run Linting and Type Checking**
+4. **Validate tests**
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
<!-- END_PIPELEX_RULES -->
diff --git a/.pipelex/inference/backends.toml b/.pipelex/inference/backends.toml
index 52fe3cbfc..2f002afc8 100644
--- a/.pipelex/inference/backends.toml
+++ b/.pipelex/inference/backends.toml
@@ -71,7 +71,7 @@ api_key = "${OPENAI_API_KEY}"
[vertexai]
display_name = "Google Vertex AI"
-enabled = true
+enabled = false # This is the only one we disable beacuse setting it up requires internet access just to get credentials so it fails in CI sandboxes
gcp_project_id = "${GCP_PROJECT_ID}"
gcp_location = "${GCP_LOCATION}"
gcp_credentials_file_path = "${GCP_CREDENTIALS_FILE_PATH}"
diff --git a/.pipelex/inference/backends/azure_openai.toml b/.pipelex/inference/backends/azure_openai.toml
index f18fb2371..2062251bb 100644
--- a/.pipelex/inference/backends/azure_openai.toml
+++ b/.pipelex/inference/backends/azure_openai.toml
@@ -97,24 +97,28 @@ model_id = "gpt-5-mini-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 0.25, output = 2.0 }
+constraints = ["temperature_must_be_1"]
[gpt-5-nano]
model_id = "gpt-5-nano-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 0.05, output = 0.4 }
+constraints = ["temperature_must_be_1"]
[gpt-5-chat]
model_id = "gpt-5-chat-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 1.25, output = 10.0 }
+constraints = ["temperature_must_be_1"]
[gpt-5]
model_id = "gpt-5-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 1.25, output = 10.0 }
+constraints = ["temperature_must_be_1"]
################################################################################
# IMAGE GENERATION MODELS
diff --git a/.pipelex/inference/routing_profiles.toml b/.pipelex/inference/routing_profiles.toml
index fa3f1d42a..0910ff285 100644
--- a/.pipelex/inference/routing_profiles.toml
+++ b/.pipelex/inference/routing_profiles.toml
@@ -94,10 +94,6 @@ default = "openai"
description = "Use Pipelex Inference backend for all its supported models"
default = "pipelex_inference"
-[profiles.all_vertexai]
-description = "Use Vertex AI backend for all its supported models"
-default = "vertexai"
-
[profiles.all_xai]
description = "Use xAI backend for all its supported models"
default = "xai"
diff --git a/.windsurfrules.md b/.windsurfrules.md
index fa12db087..211d85f6e 100644
--- a/.windsurfrules.md
+++ b/.windsurfrules.md
@@ -1,1175 +1,329 @@
<!-- BEGIN_PIPELEX_RULES -->
-## Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-### Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-### Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-#### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-#### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-#### Pipe Definitions
-
-### Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-#### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-## Single item (default)
-inputs = { document = "Text" }
-
-## Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-## Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
+# Pipelex Coding Rules
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
+## Commands
-### Structuring Models
+### Linting
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-#### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
-
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
-
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-#### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-#### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-### Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-### PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-#### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-#### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-#### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-### PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-#### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-#### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
+ After making code changes, you must always lint using `make check`.
-### PipeLLM operator
-
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-#### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-#### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-#### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-#### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-### PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-#### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
+ Always fix any issues reported by these tools before proceeding.
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
+### Running Tests
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
-### PipeCompose operator
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
+### Running Tests with Prints
-#### Basic Usage
+ If anything went wrong, you can run the tests with prints to see the error:
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-## Report Summary
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
-Based on the analysis:
-$data
+### Running specific Tests
-Generated on: {{ current_date }}
-"""
-```
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
+### Running Last Failed Tests
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
+ To rerun only the tests that failed in the previous run, use:
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
-Hi $customer.first_name,
+---
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+### Prerequisites for running command lines: activate virtual environment
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+ Do this:
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
+ or do that:
-Looking forward to moving this forward together!
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
+ (adapt the above command to the OS and available virtual environment name)
-#### Key Parameters
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-#### Template Variables
+## Coding Standards & Best Practices for Python Code
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
+This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
-### PipeImgGen operator
+### Variables, loops and indexes
-The PipeImgGen operator is used to generate images using AI image generation models.
+ - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
+ - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
+ - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
+ - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.
-#### Basic Usage
+### Enums and tests
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
+ - When defining enums related to string values, always inherit from `StrEnum`
+ - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
+ - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
+### Imports
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
+#### **Imports at the top of the file**
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
+ - Import all necessary libraries at the top of the file
+ - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
-#### Key Parameters
+- **Logging and Pretty Printing**:
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
+ - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
+ ```python
+ from pipelex import log, pretty_print
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
+ log.info("Hello, world!")
+ ```
+ - Both have a title arg which is handy when logging/printing objects:
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
+ ```python
+ log.verbose("Hello, world!", title="Your first Pipelex log")
+ pretty_print(output_object, title="Your first Pipelex output")
+ ```
+ - Both handle formatting json using Rich, pretty_print makes it prettier.
-#### Input Requirements
+- **StrEnum and Self type**:
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
+ - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import Self, StrEnum
+ ```
-The input can be named anything but must contain the prompt text for image generation.
+### Typing
-### PipeFunc operator
+#### **Always Use Type Hints**
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use Field(default_factory=...) for mutable defaults
+ - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
-#### Basic Usage
+#### **BaseModel / Pydantic Standards**
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
+ - Use `BaseModel` and respect Pydantic v2 standards
+ - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
+ - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
+ ```python
+ from pydantic import BaseModel, Field
+ from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
+
+ class MyModel(BaseModel):
+ names: list[str] = Field(default_factory=list) # OK for strings
+ numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
+ items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
+ ```
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
+### Factory Pattern
-#### Key Parameters
+ - Use factory pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
+### Error Handling
-#### Function Requirements
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
+ - NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
-The Python function must:
+### Documentation
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
+1. **Docstring Format**
```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
+ def process_image(image_path: str, size: tuple[int, int]) -> bytes:
+ """Process and resize an image.
+
+ Args:
+ image_path: Path to the source image
+ size: Tuple of (width, height) for resizing
+
+ Returns:
+ Processed image as bytes
+ """
pass
```
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-#### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-#### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-### Rules to choose LLM models used in PipeLLMs.
-
-#### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-#### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-#### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-#### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
-
-## Guide to execute a pipeline and write example code
-
-### Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-## Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-## or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-### Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-## start Pipelex
-Pipelex.make()
-## run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-### Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
+2. **Class Documentation**
+ ```python
+ class ImageProcessor:
+ """Handles image processing operations.
+
+ Provides methods for resizing, converting, and optimizing images.
+ """
+ ```
-from my_project.gantt.gantt_struct import GanttChart
+## Writing tests
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
+### Unit test generalities
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
+#### Test file structure
+- Name test files with `test_` prefix
+- Place test files in the appropriate test category directory:
+ - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+ - `tests/integration/` - for integration tests that test component interactions
+ - `tests/e2e/` - for end-to-end tests that test complete workflows
+ - `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
-## start Pipelex
-Pipelex.make()
+#### Markers
-## run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
+Apply the appropriate markers:
+- "llm: uses an LLM to generate text or objects"
+- "img_gen: uses an image generation AI"
+- "extract: uses text/image extraction from documents"
+- "inference: uses either an LLM or an image generation AI"
+- "gha_disabled: will not be able to run properly on GitHub Actions"
-### Setting up the input memory
+Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-#### Explanation of input memory
+#### Important rules
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+- Never use the unittest.mock. Use pytest-mock.
-#### Different ways to set up the input memory
+#### Test Class Structure
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
+- Always group the tests of a module into a test class:
```python
-## Here we have a single input and it's a Text.
-## If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-## Here we have a single input and it's a PDF.
-## Because PDFContent is a native concept, we can use it directly as a value,
-## the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-## Here we have a single input and it's an Image.
-## Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-## Here we have a single input, it's an image but
-## its actually a more specific concept gantt.GanttImage which refines Image,
-## so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
[email protected](loop_scope="class")
+class TestFooBar:
+ @pytest.mark.parametrize(
+ "topic, test_case_blueprint",
+ [
+ TestCases.CASE_1,
+ TestCases.CASE_2,
+ ],
)
-
-## Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
- )
-```
-
-### Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
-
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
-
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
-
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
-
- @property
- def main_stuff_as_str(self) -> str:
- ...
-
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
-
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
-
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
-
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
-
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
+ async def test_pipe_processing(
+ self,
+ request: FixtureRequest,
+ topic: str,
+ test_case_blueprint: StuffBlueprint,
+ ):
+ # Test implementation
```
-As you can see, you can extract any variable from the output working memory.
-
-#### Getting the main stuff as a specific type
-
-Simple text as a string:
-
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
+- Never more than 1 class per test module.
+- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
+- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
+#### Test Data Organization
-If it's a list, you can get a `ListContent` of the specific type.
+- If it's not already there, create a `test_data.py` file in the proper test directory
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
+### Best Practices for Testing
-or if you want, you can get the actual items as a regular python list:
+- Use strong asserts: test value, not just type and presence.
+- Use parametrize for multiple test cases
+- Test both success and failure cases
+- Verify working memory state
+- Check output structure and content
+- Use meaningful test case names
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
+- Log outputs for debugging
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
+## Writing Docs
----
+We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
-## Rules to choose LLM models used in PipeLLMs.
+### MkDocs Markdown Requirements
-### LLM Configuration System
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+## Test-Driven Development Guide
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+This document outlines our test-driven development (TDD) process and the tools available for testing.
-### LLM Handles
+### TDD Cycle
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+1. **Write a Test First**
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
+2. **Write the Code**
+ - Implement the minimum amount of code needed to pass the test
+ - Follow the project's coding standards
+ - Keep it simple - don't write more than needed
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+3. **Run Linting and Type Checking**
+4. **Validate tests**
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
<!-- END_PIPELEX_RULES -->
diff --git a/AGENTS.md b/AGENTS.md
index fa12db087..1456cbe11 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,1175 +1,299 @@
<!-- BEGIN_PIPELEX_RULES -->
-## Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-### Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-### Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-#### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-#### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-#### Pipe Definitions
-
-### Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-#### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-## Single item (default)
-inputs = { document = "Text" }
-
-## Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-## Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
-
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
-
-### Structuring Models
-
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-#### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
+# Pipelex Coding Rules
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
+## Codex Cloud Commands
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-#### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-#### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-### Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-### PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-#### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-#### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-#### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-### PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-#### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-#### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
+### Linting
-### PipeLLM operator
+ After making code changes, you must always lint using `make check`.
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-#### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-#### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-#### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-#### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-### PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-#### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
-
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
-
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
-
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
-
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
-
-### PipeCompose operator
-
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
-
-#### Basic Usage
-
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-## Report Summary
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
-Based on the analysis:
-$data
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
-Generated on: {{ current_date }}
-"""
-```
+ Always fix any issues reported by these tools before proceeding.
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
+### Running Tests in Codex Cloud
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
+ To test everything that can be tested from within the Codex Cloud sandbox, run this:
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
+ ```bash
+ make codex-tests
+ # It's equivalent to running pytest with `-m "(dry_runnable or not inference) and not (pipelex_api or codex_disabled)"`
+ # If some test fails, re-run it with `-s -vv` to see more details
+ ```
-Hi $customer.first_name,
+---
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+### Prerequisites for running command lines: activate virtual environment
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+ Do this:
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
+ or do that:
-Looking forward to moving this forward together!
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
+ (adapt the above command to the OS and available virtual environment name)
-#### Key Parameters
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-#### Template Variables
+## Coding Standards & Best Practices for Python Code
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
+This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
-### PipeImgGen operator
+### Variables, loops and indexes
-The PipeImgGen operator is used to generate images using AI image generation models.
+ - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
+ - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
+ - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
+ - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.
-#### Basic Usage
+### Enums and tests
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
+ - When defining enums related to string values, always inherit from `StrEnum`
+ - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
+ - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
+### Imports
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
+#### **Imports at the top of the file**
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
+ - Import all necessary libraries at the top of the file
+ - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
-#### Key Parameters
+- **Logging and Pretty Printing**:
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
+ - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
+ ```python
+ from pipelex import log, pretty_print
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
+ log.info("Hello, world!")
+ ```
+ - Both have a title arg which is handy when logging/printing objects:
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
+ ```python
+ log.verbose("Hello, world!", title="Your first Pipelex log")
+ pretty_print(output_object, title="Your first Pipelex output")
+ ```
+ - Both handle formatting json using Rich, pretty_print makes it prettier.
-#### Input Requirements
+- **StrEnum and Self type**:
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
+ - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import Self, StrEnum
+ ```
-The input can be named anything but must contain the prompt text for image generation.
+### Typing
-### PipeFunc operator
+#### **Always Use Type Hints**
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use Field(default_factory=...) for mutable defaults
+ - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
-#### Basic Usage
+#### **BaseModel / Pydantic Standards**
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
+ - Use `BaseModel` and respect Pydantic v2 standards
+ - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
+ - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
+ ```python
+ from pydantic import BaseModel, Field
+ from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
+
+ class MyModel(BaseModel):
+ names: list[str] = Field(default_factory=list) # OK for strings
+ numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
+ items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
+ ```
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
+### Factory Pattern
-#### Key Parameters
+ - Use factory pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
+### Error Handling
-#### Function Requirements
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
+ - NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
-The Python function must:
+### Documentation
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
+1. **Docstring Format**
```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
+ def process_image(image_path: str, size: tuple[int, int]) -> bytes:
+ """Process and resize an image.
+
+ Args:
+ image_path: Path to the source image
+ size: Tuple of (width, height) for resizing
+
+ Returns:
+ Processed image as bytes
+ """
pass
```
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-#### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-#### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-### Rules to choose LLM models used in PipeLLMs.
-
-#### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-#### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-#### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-#### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
-
-## Guide to execute a pipeline and write example code
-
-### Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-## Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-## or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-### Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-## start Pipelex
-Pipelex.make()
-## run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-### Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
+2. **Class Documentation**
+ ```python
+ class ImageProcessor:
+ """Handles image processing operations.
+
+ Provides methods for resizing, converting, and optimizing images.
+ """
+ ```
-from my_project.gantt.gantt_struct import GanttChart
+## Writing tests
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
+### Unit test generalities
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
+#### Test file structure
+- Name test files with `test_` prefix
+- Place test files in the appropriate test category directory:
+ - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+ - `tests/integration/` - for integration tests that test component interactions
+ - `tests/e2e/` - for end-to-end tests that test complete workflows
+ - `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
-## start Pipelex
-Pipelex.make()
+#### Markers
-## run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
+Apply the appropriate markers:
+- "llm: uses an LLM to generate text or objects"
+- "img_gen: uses an image generation AI"
+- "extract: uses text/image extraction from documents"
+- "inference: uses either an LLM or an image generation AI"
+- "gha_disabled: will not be able to run properly on GitHub Actions"
-### Setting up the input memory
+Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-#### Explanation of input memory
+#### Important rules
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+- Never use the unittest.mock. Use pytest-mock.
-#### Different ways to set up the input memory
+#### Test Class Structure
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
+- Always group the tests of a module into a test class:
```python
-## Here we have a single input and it's a Text.
-## If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-## Here we have a single input and it's a PDF.
-## Because PDFContent is a native concept, we can use it directly as a value,
-## the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-## Here we have a single input and it's an Image.
-## Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-## Here we have a single input, it's an image but
-## its actually a more specific concept gantt.GanttImage which refines Image,
-## so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
-
-## Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
[email protected](loop_scope="class")
+class TestFooBar:
+ @pytest.mark.parametrize(
+ "topic, test_case_blueprint",
+ [
+ TestCases.CASE_1,
+ TestCases.CASE_2,
+ ],
)
+ async def test_pipe_processing(
+ self,
+ request: FixtureRequest,
+ topic: str,
+ test_case_blueprint: StuffBlueprint,
+ ):
+ # Test implementation
```
-### Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
+- Never more than 1 class per test module.
+- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
+- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
+#### Test Data Organization
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
+- If it's not already there, create a `test_data.py` file in the proper test directory
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
+### Best Practices for Testing
- @property
- def main_stuff_as_str(self) -> str:
- ...
+- Use strong asserts: test value, not just type and presence.
+- Use parametrize for multiple test cases
+- Test both success and failure cases
+- Verify working memory state
+- Check output structure and content
+- Use meaningful test case names
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
+- Log outputs for debugging
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
+## Writing Docs
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
+We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
+### MkDocs Markdown Requirements
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
-```
-
-As you can see, you can extract any variable from the output working memory.
-
-#### Getting the main stuff as a specific type
-
-Simple text as a string:
+## Test-Driven Development Guide
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
+This document outlines our test-driven development (TDD) process and the tools available for testing.
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
+### TDD Cycle
-If it's a list, you can get a `ListContent` of the specific type.
+1. **Write a Test First**
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
-
-or if you want, you can get the actual items as a regular python list:
-
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
-
----
-
-## Rules to choose LLM models used in PipeLLMs.
-
-### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
+2. **Write the Code**
+ - Implement the minimum amount of code needed to pass the test
+ - Follow the project's coding standards
+ - Keep it simple - don't write more than needed
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+3. **Run Linting and Type Checking**
+4. **Validate tests**
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
<!-- END_PIPELEX_RULES -->
diff --git a/CLAUDE.md b/CLAUDE.md
index fa12db087..211d85f6e 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1,1175 +1,329 @@
<!-- BEGIN_PIPELEX_RULES -->
-## Guide to write or edit pipelines using the Pipelex language in .plx files
-
-- Always first write your "plan" in natural language, then transcribe it in pipelex.
-- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.
- - For a specific file: `pipelex validate path_to_file.plx`
- - For all pipelines: `pipelex validate all`
- - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.
-- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)
-
-### Pipeline File Naming
-- Files must be `.plx` for pipelines (Always add an empty line at the end of the file, and do not add trailing whitespaces to PLX files at all)
-- Files must be `.py` for code defining the data structures
-- Use descriptive names in `snake_case`
-
-### Pipeline File Outline
-A pipeline file has three main sections:
-1. Domain statement
-2. Concept definitions
-3. Pipe definitions
-
-#### Domain Statement
-```plx
-domain = "domain_name"
-description = "Description of the domain" # Optional
-```
-Note: The domain name usually matches the plx filename for single-file domains. For multi-file domains, use the subdirectory name.
-
-#### Concept Definitions
-
-Concepts represent ideas and semantic entities in your pipeline. They define what something *is*, not how it's structured.
-
-```plx
-[concept]
-ConceptName = "Description of the concept"
-```
-
-**Naming Rules:**
-- Use PascalCase for concept names
-- Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex
-- Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents
-- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page)
-
-**Native Concepts:**
-Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate.
-
-**Refining Native Concepts:**
-To create a concept that specializes a native concept without adding fields:
-
-```plx
-[concept.Landscape]
-description = "A scenic outdoor photograph"
-refines = "Image"
-```
-
-For details on how to structure concepts with fields, see the "Structuring Models" section below.
-
-#### Pipe Definitions
-
-### Pipe Base Definition
-
-```plx
-[pipe.your_pipe_name]
-type = "PipeLLM"
-description = "A description of what your pipe does"
-inputs = { input_1 = "ConceptName1", input_2 = "ConceptName2" }
-output = "ConceptName"
-```
-
-The pipes will all have at least this base definition.
-- `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition).
-So If you have this error:
-`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •
-variable='['invoice']'``
-That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.
-
-NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:
-<NEVER DO THIS>
-```plx
-inputs = {
- input_1 = "ConceptName1",
- input_2 = "ConceptName2"
-}
-```
-</NEVER DO THIS>
-
-- `output`: The name of the concept to output. The `ConceptName` should have the same name as the python class if you want structured output:
-
-#### Input Multiplicity
-
-By default, inputs expect a single item. Use bracket notation to specify multiple items:
-
-```plx
-## Single item (default)
-inputs = { document = "Text" }
-
-## Variable list - indeterminate number of items
-inputs = { documents = "Text[]" }
-
-## Fixed count - exactly N items
-inputs = { comparison_items = "Image[2]" }
-```
+# Pipelex Coding Rules
-**Key points:**
-- No brackets = single item (default behavior)
-- Use `[]` for lists of unknown length
-- Use `[N]` (where N is an integer) when operation requires exact count (e.g., comparing 2 items)
+## Commands
-### Structuring Models
+### Linting
-Once you've defined your concepts semantically (see "Concept Definitions" above), you need to specify their structure if they have fields.
-
-#### Three Ways to Structure Concepts
-
-**1. No Structure Needed**
-
-If a concept only refines a native concept without adding fields, use the TOML table syntax shown in "Concept Definitions" above. No structure section is needed.
-
-**2. Inline Structure Definition (RECOMMENDED for most cases)**
-
-For concepts with structured fields, define them inline using TOML syntax:
-
-```plx
-[concept.Invoice]
-description = "A commercial document issued by a seller to a buyer"
-
-[concept.Invoice.structure]
-invoice_number = "The unique invoice identifier" # This will be optional by default
-issue_date = { type = "date", description = "The date the invoice was issued", required = true }
-total_amount = { type = "number", description = "The total invoice amount", required = true }
-vendor_name = "The name of the vendor" # This will be optional by default
-line_items = { type = "list", item_type = "text", description = "List of items" }
-```
-
-**Supported inline field types:** `text`, `integer`, `boolean`, `number`, `date`, `list`, `dict`
-
-**Field properties:** `type`, `description`, `required` (default: false), `default_value`, `choices`, `item_type` (for lists), `key_type` and `value_type` (for dicts)
-
-**Simple syntax** (creates required text field):
-```plx
-field_name = "Field description"
-```
-
-**Detailed syntax** (with explicit properties):
-```plx
-field_name = { type = "text", description = "Field description", default_value = "default" }
-```
-
-**3. Python StructuredContent Class (For Advanced Features)**
-
-Create a Python class when you need:
-- Custom validation logic (@field_validator, @model_validator)
-- Computed properties (@property methods)
-- Custom methods or class methods
-- Complex cross-field validation
-- Reusable structures across multiple domains
-
-```python
-from pipelex.core.stuffs.structured_content import StructuredContent
-from pydantic import Field, field_validator
-
-class Invoice(StructuredContent):
- """A commercial invoice with validation."""
-
- invoice_number: str = Field(description="The unique invoice identifier")
- total_amount: float = Field(ge=0, description="The total invoice amount")
- tax_amount: float = Field(ge=0, description="Tax amount")
-
- @field_validator('tax_amount')
- @classmethod
- def validate_tax(cls, v, info):
- """Ensure tax doesn't exceed total."""
- total = info.data.get('total_amount', 0)
- if v > total:
- raise ValueError('Tax amount cannot exceed total amount')
- return v
-```
-
-**Location:** Create models in `my_project/some_domain/some_domain_struct.py`. Classes inheriting from `StructuredContent` are automatically discovered.
-
-#### Decision Rules for Agents
-
-**If concept already exists:**
-- If it's already inline → KEEP IT INLINE unless user explicitly asks to convert or features require Python class
-- If it's already a Python class → KEEP IT as Python class
-
-**If creating new concept:**
-1. Does it only refine a native concept without adding fields? → Use concept-only declaration
-2. Does it need custom validation, computed properties, or methods? → Use Python class
-3. Otherwise → Use inline structure (fastest and simplest)
-
-**When to suggest conversion to Python class:**
-- User needs validation logic beyond type checking
-- User needs computed properties or custom methods
-- Structure needs to be reused across multiple domains
-- Complex type relationships or inheritance required
-
-#### Inline Structure Limitations
-
-Inline structures:
-- ✅ Support all common field types (text, number, date, list, dict, etc.)
-- ✅ Support required/optional fields, defaults, choices
-- ✅ Generate full Pydantic models with validation
-- ❌ Cannot have custom validators or complex validation logic
-- ❌ Cannot have computed properties or custom methods
-- ❌ Cannot refine custom (non-native) concepts
-- ❌ Limited IDE autocomplete compared to explicit Python classes
-
-
-### Pipe Controllers and Pipe Operators
-
-Look at the Pipes we have in order to adapt it. Pipes are organized in two categories:
-
-1. **Controllers** - For flow control:
- - `PipeSequence` - For creating a sequence of multiple steps
- - `PipeCondition` - If the next pipe depends of the expression of a stuff in the working memory
- - `PipeParallel` - For parallelizing pipes
-
-2. **Operators** - For specific tasks:
- - `PipeLLM` - Generate Text and Objects (include Vision LLM)
- - `PipeExtract` - Extract text and images from an image or a PDF
- - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.
- - `PipeImgGen` - Generate Images
- - `PipeFunc` - For running classic python scripts
-
-### PipeSequence controller
-
-Purpose: PipeSequence executes multiple pipes in a defined order, where each step can use results from original inputs or from previous steps.
-
-#### Basic Definition
-```plx
-[pipe.your_sequence_name]
-type = "PipeSequence"
-description = "Description of what this sequence does"
-inputs = { input_name = "InputType" } # All the inputs of the sub pipes, except the ones generated by intermediate steps
-output = "OutputType"
-steps = [
- { pipe = "first_pipe", result = "first_result" },
- { pipe = "second_pipe", result = "second_result" },
- { pipe = "final_pipe", result = "final_result" }
-]
-```
-
-#### Key Components
-
-1. **Steps Array**: List of pipes to execute in sequence
- - `pipe`: Name of the pipe to execute
- - `result`: Name to assign to the pipe's output that will be in the working memory
-
-#### Using PipeBatch in Steps
-
-You can use PipeBatch functionality within steps using `batch_over` and `batch_as`:
-
-```plx
-steps = [
- { pipe = "process_items", batch_over = "input_list", batch_as = "current_item", result = "processed_items"
- }
-]
-```
-
-1. **batch_over**: Specifies a `ListContent` field to iterate over. Each item in the list will be processed individually and IN PARALLEL by the pipe.
- - Must be a `ListContent` type containing the items to process
- - Can reference inputs or results from previous steps
-
-2. **batch_as**: Defines the name that will be used to reference the current item being processed
- - This name can be used in the pipe's input mappings
- - Makes each item from the batch available as a single element
-
-The result of a batched step will be a `ListContent` containing the outputs from processing each item.
-
-### PipeCondition controller
-
-The PipeCondition controller allows you to implement conditional logic in your pipeline, choosing which pipe to execute based on an evaluated expression. It supports both direct expressions and expression templates.
-
-#### Basic usage
-
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression = "input_data.category"
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-or
-```plx
-[pipe.conditional_operation]
-type = "PipeCondition"
-description = "A conditional pipe to decide whether..."
-inputs = { input_data = "CategoryInput" }
-output = "native.Text"
-expression_template = "{{ input_data.category }}" # Jinja2 code
-default_outcome = "process_medium"
-
-[pipe.conditional_operation.outcomes]
-small = "process_small"
-medium = "process_medium"
-large = "process_large"
-```
-
-#### Key Parameters
-
-- `expression`: Direct boolean or string expression (mutually exclusive with expression_template)
-- `expression_template`: Jinja2 template for more complex conditional logic (mutually exclusive with expression)
-- `outcomes`: Dictionary mapping expression results to pipe codes:
- 1. The key on the left (`small`, `medium`) is the result of `expression` or `expression_template`
- 2. The value on the right (`process_small`, `process_medium`, etc.) is the name of the pipe to trigger
-- `default_outcome`: **Required** - The pipe to execute if the expression doesn't match any key in outcomes. Use `"fail"` if you want the pipeline to fail when no match is found
-
-Example with fail as default:
-```plx
-[pipe.strict_validation]
-type = "PipeCondition"
-description = "Validate with strict matching"
-inputs = { status = "Status" }
-output = "Text"
-expression = "status.value"
-default_outcome = "fail"
-
-[pipe.strict_validation.outcomes]
-approved = "process_approved"
-rejected = "process_rejected"
-```
+ After making code changes, you must always lint using `make check`.
-### PipeLLM operator
-
-PipeLLM is used to:
-1. Generate text or objects with LLMs
-2. Process images with Vision LLMs
-
-#### Basic Usage
-
-Simple Text Generation:
-```plx
-[pipe.write_story]
-type = "PipeLLM"
-description = "Write a short story"
-output = "Text"
-prompt = """
-Write a short story about a programmer.
-"""
-```
-
-Structured Data Extraction:
-```plx
-[pipe.extract_info]
-type = "PipeLLM"
-description = "Extract information"
-inputs = { text = "Text" }
-output = "PersonInfo"
-prompt = """
-Extract person information from this text:
-@text
-"""
-```
-
-Supports system instructions:
-```plx
-[pipe.expert_analysis]
-type = "PipeLLM"
-description = "Expert analysis"
-output = "Analysis"
-system_prompt = "You are a data analysis expert"
-prompt = "Analyze this data"
-```
-
-#### Multiple Outputs
-
-Generate multiple outputs (fixed number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[3]" # Generate exactly 3 ideas
-prompt = "Generate 3 ideas"
-```
-
-Generate multiple outputs (variable number) - use bracket notation:
-```plx
-[pipe.generate_ideas]
-type = "PipeLLM"
-description = "Generate ideas"
-output = "Idea[]" # Let the LLM decide how many to generate
-prompt = "Generate ideas"
-```
-
-#### Vision
-
-Process images with VLMs (image inputs must be tagged in the prompt):
-```plx
-[pipe.analyze_image]
-type = "PipeLLM"
-description = "Analyze image"
-inputs = { image = "Image" }
-output = "ImageAnalysis"
-prompt = """
-Describe what you see in this image:
-
-$image
-"""
-```
-
-You can also reference images inline in meaningful sentences to guide the Visual LLM:
-```plx
-[pipe.compare_images]
-type = "PipeLLM"
-description = "Compare two images"
-inputs = { photo = "Image", painting = "Image" }
-output = "Analysis"
-prompt = "Analyze the colors in $photo and the shapes in $painting."
-```
-
-#### Writing prompts for PipeLLM
-
-**Insert stuff inside a tagged block**
-
-If the inserted text is supposedly a long text, made of several lines or paragraphs, you want it inserted inside a block, possibly a block tagged and delimlited with proper syntax as one would do in a markdown documentation. To include stuff as a block, use the "@" prefix.
-
-Example template:
-```plx
-prompt = """
-Match the expense with its corresponding invoice:
-
-@expense
-
-@invoices
-"""
-```
-In the example above, the expense data and the invoices data are obviously made of several lines each, that's why it makes sense to use the "@" prefix in order to have them delimited inside a block. Note that our preprocessor will automatically include the block's title, so it doesn't need to be explicitly written in the prompt.
-
-DO NOT write things like "Here is the expense: @expense".
-DO write simply "@expense" alone in an isolated line.
-
-**Insert stuff inline**
-
-If the inserted text is short text and it makes sense to have it inserted directly into a sentence, you want it inserted inline. To insert stuff inline, use the "$" prefix. This will insert the stuff without delimiters and the content will be rendered as plain text.
-
-Example template:
-```plx
-prompt = """
-Your goal is to summarize everything related to $topic in the provided text:
-
-@text
-
-Please provide only the summary, with no additional text or explanations.
-Your summary should not be longer than 2 sentences.
-"""
-```
-
-In the example above, $topic will be inserted inline, whereas @text will be a a delimited block.
-Be sure to make the proper choice of prefix for each insertion.
-
-DO NOT write "$topic" alone in an isolated line.
-DO write things like "Write an essay about $topic" to include text into an actual sentence.
-
-
-### PipeExtract operator
-
-The PipeExtract operator is used to extract text and images from an image or a PDF
-
-#### Simple Text Extraction
-```plx
-[pipe.extract_info]
-type = "PipeExtract"
-description = "extract the information"
-inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.
-output = "Page"
-```
-
-Using Extract Model Settings:
-```plx
-[pipe.extract_with_model]
-type = "PipeExtract"
-description = "Extract with specific model"
-inputs = { document = "PDF" }
-output = "Page"
-model = "base_extract_mistral" # Use predefined extract preset or model alias
-```
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
-Only one input is allowed and it must either be an `Image` or a `PDF`. The input can be named anything.
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
-The output concept `Page` is a native concept, with the structure `PageContent`:
-It corresponds to 1 page. Therefore, the PipeExtract is outputing a `ListContent` of `Page`
+ Always fix any issues reported by these tools before proceeding.
-```python
-class TextAndImagesContent(StuffContent):
- text: TextContent | None
- images: list[ImageContent] | None
+### Running Tests
-class PageContent(StructuredContent): # CONCEPT IS "Page"
- text_and_images: TextAndImagesContent
- page_view: ImageContent | None = None
-```
-- `text_and_images` are the text, and the related images found in the input image or PDF.
-- `page_view` is the screenshot of the whole pdf page/image.
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
-### PipeCompose operator
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
-The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.
+### Running Tests with Prints
-#### Basic Usage
+ If anything went wrong, you can run the tests with prints to see the error:
-Simple Template Composition:
-```plx
-[pipe.compose_report]
-type = "PipeCompose"
-description = "Compose a report using template"
-inputs = { data = "ReportData" }
-output = "Text"
-template = """
-## Report Summary
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
-Based on the analysis:
-$data
+### Running specific Tests
-Generated on: {{ current_date }}
-"""
-```
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
-Using Named Templates:
-```plx
-[pipe.use_template]
-type = "PipeCompose"
-description = "Use a predefined template"
-inputs = { content = "Text" }
-output = "Text"
-template_name = "standard_report_template"
-```
+### Running Last Failed Tests
-Using Nested Template Section (for more control):
-```plx
-[pipe.advanced_template]
-type = "PipeCompose"
-description = "Use advanced template settings"
-inputs = { data = "ReportData" }
-output = "Text"
-
-[pipe.advanced_template.template]
-template = "Report: $data"
-category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-```
+ To rerun only the tests that failed in the previous run, use:
-CRM Email Template:
-```plx
-[pipe.compose_follow_up_email]
-type = "PipeCompose"
-description = "Compose a personalized follow-up email for CRM"
-inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }
-output = "Text"
-template_category = "html"
-templating_style = { tag_style = "square_brackets", text_format = "html" }
-template = """
-Subject: Following up on our $deal.product_name discussion
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
-Hi $customer.first_name,
+---
-I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.
+### Prerequisites for running command lines: activate virtual environment
-Based on our discussion, I understand that your key requirements are: $deal.customer_requirements
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
-I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:
+ Do this:
-**Next Steps:**
-- Schedule a demo tailored to your specific needs
-- Provide you with a customized quote based on your requirements
-- Connect you with our implementation team
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
-Would you be available for a 30-minute call this week? I have openings on:
-{% for slot in available_slots %}
-- {{ slot }}
-{% endfor %}
+ or do that:
-Looking forward to moving this forward together!
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
-Best regards,
-$sales_rep.name
-$sales_rep.title
-$sales_rep.phone | $sales_rep.email
-"""
-```
+ (adapt the above command to the OS and available virtual environment name)
-#### Key Parameters
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
-- `template`: Inline template string (mutually exclusive with template_name)
-- `template_name`: Name of a predefined template (mutually exclusive with template)
-- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)
-- `templating_style`: Styling options for template rendering
-- `extra_context`: Additional context variables for template
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
-For more control, you can use a nested `template` section instead of the `template` field:
-- `template.template`: The template string
-- `template.category`: Template type
-- `template.templating_style`: Styling options
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-#### Template Variables
+## Coding Standards & Best Practices for Python Code
-Use the same variable insertion rules as PipeLLM:
-- `@variable` for block insertion (multi-line content)
-- `$variable` for inline insertion (short text)
+This document outlines the core coding standards, best practices, and quality control procedures for the codebase.
-### PipeImgGen operator
+### Variables, loops and indexes
-The PipeImgGen operator is used to generate images using AI image generation models.
+ - Variable names should have a minimum length of 3 characters. No exceptions: name your `for` loop indexes like `index_foobar`, your exceptions `exc` or more specific like `validation_error` when there are several layers of exceptions, and use `for key, value in ...` for key/value pairs.
+ - When looping on the keys of a dict, use `for key in the_dict` rather than `for key in the_dict.keys()` otherwise you won't pass linting.
+ - Avoid inline for loops, unless it's ultra-simple and holds on oneline.
+ - If you have a variable that will get its value differently through different code paths, declare it first with a type, e.g. `pipe_code: str` but DO NOT give it a default value like `pipe_code: str = ""` unless it's really justified. We want the variable to be unbound until all paths are covered, and the linters will help us avoid bugs this way.
-#### Basic Usage
+### Enums and tests
-Simple Image Generation:
-```plx
-[pipe.generate_image]
-type = "PipeImgGen"
-description = "Generate an image from prompt"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-```
+ - When defining enums related to string values, always inherit from `StrEnum`
+ - Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
+ - As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
-Using Image Generation Settings:
-```plx
-[pipe.generate_photo]
-type = "PipeImgGen"
-description = "Generate a high-quality photo"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Photo"
-model = { model = "fast-img-gen" }
-aspect_ratio = "16:9"
-quality = "hd"
-```
+### Imports
-Multiple Image Generation:
-```plx
-[pipe.generate_variations]
-type = "PipeImgGen"
-description = "Generate multiple image variations"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image[3]"
-seed = "auto"
-```
+#### **Imports at the top of the file**
-Advanced Configuration:
-```plx
-[pipe.generate_custom]
-type = "PipeImgGen"
-description = "Generate image with custom settings"
-inputs = { prompt = "ImgGenPrompt" }
-output = "Image"
-model = "img_gen_preset_name" # Use predefined preset
-aspect_ratio = "1:1"
-quality = "hd"
-background = "transparent"
-output_format = "png"
-is_raw = false
-safety_tolerance = 3
-```
+ - Import all necessary libraries at the top of the file
+ - Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
-#### Key Parameters
+- **Logging and Pretty Printing**:
-**Image Generation Settings:**
-- `model`: Model choice (preset name or inline settings with model name)
-- `quality`: Image quality ("standard", "hd")
+ - Both `log()` and `pretty_print()` can be imported from `pipelex` directly:
+ ```python
+ from pipelex import log, pretty_print
-**Output Configuration:**
-- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)
-- `output_format`: File format ("png", "jpeg", "webp")
-- `background`: Background type ("default", "transparent")
+ log.info("Hello, world!")
+ ```
+ - Both have a title arg which is handy when logging/printing objects:
-**Generation Control:**
-- `seed`: Random seed (integer or "auto")
-- `is_raw`: Whether to apply post-processing
-- `is_moderated`: Enable content moderation
-- `safety_tolerance`: Content safety level (1-6)
+ ```python
+ log.verbose("Hello, world!", title="Your first Pipelex log")
+ pretty_print(output_object, title="Your first Pipelex output")
+ ```
+ - Both handle formatting json using Rich, pretty_print makes it prettier.
-#### Input Requirements
+- **StrEnum and Self type**:
-PipeImgGen requires exactly one input that must be either:
-- An `ImgGenPrompt` concept
-- A concept that refines `ImgGenPrompt`
+ - Both `StrEnum` and `Self` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import Self, StrEnum
+ ```
-The input can be named anything but must contain the prompt text for image generation.
+### Typing
-### PipeFunc operator
+#### **Always Use Type Hints**
-The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.
+ - Every function parameter must be typed
+ - Every function return must be typed
+ - Use type hints for all variables where type is not obvious
+ - Use dict, list, tuple types with lowercase first letter: dict[], list[], tuple[]
+ - Use type hints for all fields
+ - Use Field(default_factory=...) for mutable defaults
+ - Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
-#### Basic Usage
+#### **BaseModel / Pydantic Standards**
-Simple Function Call:
-```plx
-[pipe.process_data]
-type = "PipeFunc"
-description = "Process data using custom function"
-inputs = { input_data = "DataType" }
-output = "ProcessedData"
-function_name = "process_data_function"
-```
+ - Use `BaseModel` and respect Pydantic v2 standards
+ - Use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`
+ - Keep models focused and single-purpose
+ - For list fields with non-string items in BaseModels, use `empty_list_factory_of()` to avoid linter complaints:
+ ```python
+ from pydantic import BaseModel, Field
+ from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
+
+ class MyModel(BaseModel):
+ names: list[str] = Field(default_factory=list) # OK for strings
+ numbers: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")
+ items: list[MyItem] = Field(default_factory=empty_list_factory_of(MyItem), description="A list of items")
+ ```
-File Processing Example:
-```plx
-[pipe.read_file]
-type = "PipeFunc"
-description = "Read file content"
-inputs = { file_path = "FilePath" }
-output = "FileContent"
-function_name = "read_file_content"
-```
+### Factory Pattern
-#### Key Parameters
+ - Use factory pattern for object creation when dealing with multiple implementations
+ - Our factory methods are named `make_from_...` and such
-- `function_name`: Name of the Python function to call (must be registered in func_registry)
+### Error Handling
-#### Function Requirements
+ - Always catch exceptions at the place where you can add useful context to it.
+ - Use try/except blocks with specific exceptions
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
+ - NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
+
+ ```python
+ try:
+ self.models_manager.setup()
+ except RoutingProfileLibraryNotFoundError as exc:
+ msg = "The routing library could not be found, please call `pipelex init config` to create it"
+ raise PipelexSetupError(msg) from exc
+ ```
-The Python function must:
+### Documentation
-1. **Be registered** in the `func_registry`
-2. **Accept `working_memory`** as a parameter:
+1. **Docstring Format**
```python
- async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:
- # Function implementation
+ def process_image(image_path: str, size: tuple[int, int]) -> bytes:
+ """Process and resize an image.
+
+ Args:
+ image_path: Path to the source image
+ size: Tuple of (width, height) for resizing
+
+ Returns:
+ Processed image as bytes
+ """
pass
```
-3. **Return appropriate types**:
- - `StuffContent`: Single content object
- - `list[StuffContent]`: Multiple content objects (becomes ListContent)
- - `str`: Simple string (becomes TextContent)
-
-#### Function Registration
-
-Functions must be registered in the function registry before use:
-
-```python
-from pipelex.system.registries.func_registry import func_registry
-
-@func_registry.register("my_function_name")
-async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:
- # Access inputs from working memory
- input_data = working_memory.get_stuff("input_name")
-
- # Process data
- result = process_logic(input_data.content)
-
- # Return result
- return MyResultContent(data=result)
-```
-
-#### Working Memory Access
-
-Inside the function, access pipeline inputs through working memory:
-
-```python
-async def process_function(working_memory: WorkingMemory) -> TextContent:
- # Get input stuff by name
- input_stuff = working_memory.get_stuff("input_name")
-
- # Access the content
- input_content = input_stuff.content
-
- # Process and return
- processed_text = f"Processed: {input_content.text}"
- return TextContent(text=processed_text)
-```
-
----
-
-### Rules to choose LLM models used in PipeLLMs.
-
-#### LLM Configuration System
-
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
-
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
-
-#### LLM Handles
-
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
-
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-#### Using an LLM Handle in a PipeLLM
-
-Here is an example of using a model to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-#### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
-
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
-
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
-
----
-
-ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.
-- For a specific bundle/file: `pipelex validate path_to_file.plx`
-- For all pipelines: `pipelex validate all`
-- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.
-
-Then, create an example file to run the pipeline in the `examples` folder.
-But don't write documentation unless asked explicitly to.
-
-## Guide to execute a pipeline and write example code
-
-### Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-## Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-## or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
-### Example to execute a pipeline with text output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-
-
-async def hello_world() -> str:
- """
- This function demonstrates the use of a super simple Pipelex pipeline to generate text.
- """
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="hello_world",
- )
-
- return pipe_output.main_stuff_as_str
-
-
-## start Pipelex
-Pipelex.make()
-## run sample using asyncio
-output_text = asyncio.run(hello_world())
-pretty_print(output_text, title="Your first Pipelex output")
-```
-
-### Example to execute a pipeline with structured output
-
-```python
-import asyncio
-
-from pipelex import pretty_print
-from pipelex.pipelex import Pipelex
-from pipelex.pipeline.execute import execute_pipeline
-from pipelex.core.stuffs.image_content import ImageContent
+2. **Class Documentation**
+ ```python
+ class ImageProcessor:
+ """Handles image processing operations.
+
+ Provides methods for resizing, converting, and optimizing images.
+ """
+ ```
-from my_project.gantt.gantt_struct import GanttChart
+## Writing tests
-SAMPLE_NAME = "extract_gantt"
-IMAGE_URL = "assets/gantt/gantt_tree_house.png"
+### Unit test generalities
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
-async def extract_gantt(image_url: str) -> GanttChart:
- # Run the pipe
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
- )
- # Output the result
- return pipe_output.main_stuff_as(content_type=GanttChart)
+#### Test file structure
+- Name test files with `test_` prefix
+- Place test files in the appropriate test category directory:
+ - `tests/unit/` - for unit tests that test individual functions/classes in isolation
+ - `tests/integration/` - for integration tests that test component interactions
+ - `tests/e2e/` - for end-to-end tests that test complete workflows
+ - `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
-## start Pipelex
-Pipelex.make()
+#### Markers
-## run sample using asyncio
-gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))
-pretty_print(gantt_chart, title="Gantt Chart")
-```
+Apply the appropriate markers:
+- "llm: uses an LLM to generate text or objects"
+- "img_gen: uses an image generation AI"
+- "extract: uses text/image extraction from documents"
+- "inference: uses either an LLM or an image generation AI"
+- "gha_disabled: will not be able to run properly on GitHub Actions"
-### Setting up the input memory
+Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.
-#### Explanation of input memory
+#### Important rules
-The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:
-```python
-StuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str
-PipelineInputs = dict[str, StuffContentOrData]
-```
-As you can seen, we made it so different ways can be used to define that stuff using structured content or data.
+- Never use the unittest.mock. Use pytest-mock.
-#### Different ways to set up the input memory
+#### Test Class Structure
-So here are a few concrete examples of calls to execute_pipeline with various ways to set up the input memory:
+- Always group the tests of a module into a test class:
```python
-## Here we have a single input and it's a Text.
-## If you assign a string, by default it will be considered as a TextContent.
- pipe_output = await execute_pipeline(
- pipe_code="master_advisory_orchestrator",
- inputs={
- "user_input": problem_description,
- },
- )
-
-## Here we have a single input and it's a PDF.
-## Because PDFContent is a native concept, we can use it directly as a value,
-## the system knows what content it corresponds to:
- pipe_output = await execute_pipeline(
- pipe_code="power_extractor_dpe",
- inputs={
- "document": PDFContent(url=pdf_url),
- },
- )
-
-## Here we have a single input and it's an Image.
-## Because ImageContent is a native concept, we can use it directly as a value:
- pipe_output = await execute_pipeline(
- pipe_code="fashion_variation_pipeline",
- inputs={
- "fashion_photo": ImageContent(url=image_url),
- },
- )
-
-## Here we have a single input, it's an image but
-## its actually a more specific concept gantt.GanttImage which refines Image,
-## so we must provide it using a dict with the concept and the content:
- pipe_output = await execute_pipeline(
- pipe_code="extract_gantt_by_steps",
- inputs={
- "gantt_chart_image": {
- "concept": "gantt.GanttImage",
- "content": ImageContent(url=image_url),
- }
- },
[email protected](loop_scope="class")
+class TestFooBar:
+ @pytest.mark.parametrize(
+ "topic, test_case_blueprint",
+ [
+ TestCases.CASE_1,
+ TestCases.CASE_2,
+ ],
)
-
-## Here is a more complex example with multiple inputs assigned using different ways:
- pipe_output = await execute_pipeline(
- pipe_code="retrieve_then_answer",
- dynamic_output_concept_code="contracts.Fees",
- inputs={
- "text": load_text_from_path(path=text_path),
- "question": {
- "concept": "answer.Question",
- "content": question,
- },
- "client_instructions": client_instructions,
- },
- )
-```
-
-### Using the outputs of a pipeline
-
-All pipe executions return a `PipeOutput` object.
-It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.
-It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:
-
-```python
-
-class PipeOutput(BaseModel):
- working_memory: WorkingMemory = Field(default_factory=WorkingMemory)
- pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)
-
- @property
- def main_stuff(self) -> Stuff:
- ...
-
- def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:
- ...
-
- def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:
- ...
-
- def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:
- ...
-
- @property
- def main_stuff_as_text(self) -> TextContent:
- ...
-
- @property
- def main_stuff_as_str(self) -> str:
- ...
-
- @property
- def main_stuff_as_image(self) -> ImageContent:
- ...
-
- @property
- def main_stuff_as_text_and_image(self) -> TextAndImagesContent:
- ...
-
- @property
- def main_stuff_as_number(self) -> NumberContent:
- ...
-
- @property
- def main_stuff_as_html(self) -> HtmlContent:
- ...
-
- @property
- def main_stuff_as_mermaid(self) -> MermaidContent:
- ...
+ async def test_pipe_processing(
+ self,
+ request: FixtureRequest,
+ topic: str,
+ test_case_blueprint: StuffBlueprint,
+ ):
+ # Test implementation
```
-As you can see, you can extract any variable from the output working memory.
-
-#### Getting the main stuff as a specific type
-
-Simple text as a string:
-
-```python
-result = pipe_output.main_stuff_as_str
-```
-Structured object (BaseModel):
+- Never more than 1 class per test module.
+- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
+- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
-```python
-result = pipe_output.main_stuff_as(content_type=GanttChart)
-```
+#### Test Data Organization
-If it's a list, you can get a `ListContent` of the specific type.
+- If it's not already there, create a `test_data.py` file in the proper test directory
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
-```python
-result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)
-```
+### Best Practices for Testing
-or if you want, you can get the actual items as a regular python list:
+- Use strong asserts: test value, not just type and presence.
+- Use parametrize for multiple test cases
+- Test both success and failure cases
+- Verify working memory state
+- Check output structure and content
+- Use meaningful test case names
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
+- Log outputs for debugging
-```python
-result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)
-```
+## Writing Docs
----
+We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
-## Rules to choose LLM models used in PipeLLMs.
+### MkDocs Markdown Requirements
-### LLM Configuration System
+- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.
-In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.
-LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:
+## Test-Driven Development Guide
-- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`
-- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`
-- **Routing**: `.pipelex/inference/routing_profiles.toml`
+This document outlines our test-driven development (TDD) process and the tools available for testing.
-### LLM Handles
+### TDD Cycle
-An llm_handle can be either:
-1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system
-2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:
+1. **Write a Test First**
-```toml
-[aliases]
-base-claude = "claude-4.5-sonnet"
-base-gpt = "gpt-5"
-base-gemini = "gemini-2.5-flash"
-base-mistral = "mistral-medium"
-```
-
-The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.
-
-### Using an LLM Handle in a PipeLLM
-
-Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:
-
-```plx
-[pipe.hello_world]
-type = "PipeLLM"
-description = "Write text about Hello World."
-output = "Text"
-model = { model = "gpt-5", temperature = 0.9 }
-prompt = """
-Write a haiku about Hello World.
-"""
-```
-
-As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").
-
-### LLM Presets
-
-Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.
-
-Examples:
-```toml
-llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
-```
-
-The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
-
-```plx
-[pipe.extract_invoice]
-type = "PipeLLM"
-description = "Extract invoice information from an invoice text transcript"
-inputs = { invoice_text = "InvoiceText" }
-output = "Invoice"
-model = "llm_to_extract_invoice"
-prompt = """
-Extract invoice information from this invoice:
-
-The category of this invoice is: $invoice_details.category.
-
-@invoice_text
-"""
-```
+2. **Write the Code**
+ - Implement the minimum amount of code needed to pass the test
+ - Follow the project's coding standards
+ - Keep it simple - don't write more than needed
-The setting here `model = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.
-You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.
+3. **Run Linting and Type Checking**
+4. **Validate tests**
-You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
<!-- END_PIPELEX_RULES -->
diff --git a/Makefile b/Makefile
index d0b97edec..a8fc7c13f 100644
--- a/Makefile
+++ b/Makefile
@@ -20,7 +20,7 @@ VENV_PIPELEX_DEV := "$(VIRTUAL_ENV)/bin/pipelex-dev"
UV_MIN_VERSION = $(shell grep -m1 'required-version' pyproject.toml | sed -E 's/.*= *"([^<>=, ]+).*/\1/')
-USUAL_PYTEST_MARKERS := "(dry_runnable or not (inference or llm or img_gen or extract)) and not (needs_output or pipelex_api)"
+USUAL_PYTEST_MARKERS := "(dry_runnable or not (inference or llm or img_gen or extract)) and not pipelex_api"
define PRINT_TITLE
$(eval PROJECT_PART := [$(PROJECT_NAME)])
@@ -54,9 +54,11 @@ make lint - lint with ruff check
make pyright - Check types with pyright
make mypy - Check types with mypy
+make rules - Install agent rules for contributing to Pipelex
make up-kit-configs - Update kit configs from .pipelex/
make ukc - Shorthand -> up-kit-configs
make check-config-sync - Verify .pipelex and pipelex/kit/configs are in sync
+make check-rules - Verify installed agent rules match kit templates
make ccs - Shorthand -> check-config-sync
make cleanenv - Remove virtual env and lock files
@@ -112,6 +114,7 @@ export HELP
.PHONY: \
all help env lock install update build \
format lint pyright mypy pylint \
+ rules up-kit-configs ukc check-config-sync check-rules ccs \
cleanderived cleanenv cleanall \
test test-xdist t test-quiet tq test-with-prints tp test-inference ti \
test-llm tl test-img-gen tg test-extract te codex-tests gha-tests \
@@ -119,7 +122,6 @@ export HELP
validate v check c cc \
merge-check-ruff-lint merge-check-ruff-format merge-check-mypy merge-check-pyright \
li check-unused-imports fix-unused-imports check-uv check-TODOs docs docs-check docs-deploy \
- up-kit-configs ukc check-config-sync ccs \
test-count check-test-badge
all help:
@@ -174,6 +176,14 @@ build: env
$(call PRINT_TITLE,"Building the wheels")
@uv build
+rules: env
+ $(call PRINT_TITLE,"Installing agent rules for contributing to Pipelex")
+ $(VENV_PIPELEX) kit rules --set coding_standards
+
+check-rules: env
+ $(call PRINT_TITLE,"Checking installed agent rules against templates")
+ $(VENV_PIPELEX_DEV) check-rules --quiet
+
up-kit-configs:
$(call PRINT_TITLE,"Updating kit configs from .pipelex/")
@rsync -av --delete .pipelex/ pipelex/kit/configs/
@@ -183,7 +193,7 @@ ukc: up-kit-configs
check-config-sync: env
$(call PRINT_TITLE,"Checking config sync between .pipelex and pipelex/kit/configs")
- $(VENV_PIPELEX_DEV) check-config-sync
+ $(VENV_PIPELEX_DEV) check-config-sync --quiet
ccs: check-config-sync
@echo "> done: ccs = check-config-sync"
@@ -227,7 +237,7 @@ cleanall: cleanderived cleanenv cleanconfig
codex-tests: env
$(call PRINT_TITLE,"Unit testing for Codex")
@echo "• Running unit tests for Codex (excluding inference and codex_disabled)"
- $(VENV_PYTEST) --exitfirst -m "(dry_runnable or not inference) and not (needs_output or pipelex_api or codex_disabled)" || [ $$? = 5 ]
+ $(VENV_PYTEST) -n auto --exitfirst -m "(dry_runnable or not inference) and not (pipelex_api or codex_disabled)" || [ $$? = 5 ]
gha-tests: env
$(call PRINT_TITLE,"Unit testing for github actions")
@@ -253,7 +263,11 @@ test: env
$(call PRINT_TITLE,"Unit testing without prints but displaying logs via pytest for WARNING level and above")
@echo "• Running unit tests"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -262,7 +276,11 @@ test-xdist: env
$(call PRINT_TITLE,"Unit testing without prints but displaying logs via pytest for WARNING level and above")
@echo "• Running unit tests"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) -n auto -m $(USUAL_PYTEST_MARKERS) -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) -n auto -m $(USUAL_PYTEST_MARKERS) -o log_level=WARNING --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) -n auto -m $(USUAL_PYTEST_MARKERS) -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) -n auto -m $(USUAL_PYTEST_MARKERS) -o log_level=WARNING $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -274,7 +292,11 @@ test-quiet: env
$(call PRINT_TITLE,"Unit testing without prints but displaying logs via pytest for WARNING level and above")
@echo "• Running unit tests"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) -m $(USUAL_PYTEST_MARKERS) -o log_cli=true -o log_level=WARNING $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -286,7 +308,11 @@ test-with-prints: env
$(call PRINT_TITLE,"Unit testing with prints and our rich logs")
@echo "• Running unit tests"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) -s -m $(USUAL_PYTEST_MARKERS) $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -302,7 +328,11 @@ tb: env
test-inference-with-prints: env
$(call PRINT_TITLE,"Unit testing")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --pipe-run-mode live -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --pipe-run-mode live -m "inference" -s -rfE --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --pipe-run-mode live -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --pipe-run-mode live -m "inference" -s -rfE $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -310,7 +340,11 @@ test-inference-with-prints: env
test-inference-fast: env
$(call PRINT_TITLE,"Unit testing")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) -n auto --pipe-run-mode live -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) -n auto --pipe-run-mode live -m "inference" -s -rfE --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) -n auto --pipe-run-mode live -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) -n auto --pipe-run-mode live -m "inference" -s -rfE $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -324,7 +358,11 @@ ti: test-inference-fast
ti-dry: env
$(call PRINT_TITLE,"Unit testing")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --pipe-run-mode dry --exitfirst -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --pipe-run-mode dry --exitfirst -m "inference" -s -rfE --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --pipe-run-mode dry --exitfirst -m "inference" -s -rfE -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --pipe-run-mode dry --exitfirst -m "inference" -s -rfE $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -332,7 +370,11 @@ ti-dry: env
test-llm: env
$(call PRINT_TITLE,"Unit testing LLM")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "llm" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "llm" -s --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "llm" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "llm" -s $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -343,7 +385,11 @@ tl: test-llm
test-extract: env
$(call PRINT_TITLE,"Unit testing Extract")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "extract" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "extract" -s --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "extract" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "extract" -s $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -354,7 +400,11 @@ te: test-extract
test-img-gen: env
$(call PRINT_TITLE,"Unit testing Image Generation")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "img_gen" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "img_gen" -s --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "img_gen" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --pipe-run-mode live --exitfirst -m "img_gen" -s $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -365,7 +415,11 @@ tg: test-img-gen
test-pipelex-api: env
$(call PRINT_TITLE,"Unit testing")
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --exitfirst -m "pipelex_api" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --exitfirst -m "pipelex_api" -s --lf $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ else \
+ $(VENV_PYTEST) --exitfirst -m "pipelex_api" -s -k "$(TEST)" $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
+ fi; \
else \
$(VENV_PYTEST) --exitfirst -m "pipelex_api" -s $(if $(filter 1,$(VERBOSE)),-v,$(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,))); \
fi
@@ -377,7 +431,11 @@ cov: env
$(call PRINT_TITLE,"Unit testing with coverage")
@echo "• Running unit tests with coverage"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --lf $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ else \
+ $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ fi; \
else \
$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
fi
@@ -386,7 +444,11 @@ cov-missing: env
$(call PRINT_TITLE,"Unit testing with coverage and missing lines")
@echo "• Running unit tests with coverage and missing lines"
@if [ -n "$(TEST)" ]; then \
- $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ if [ "$(TEST)" = "LF" ] || [ "$(TEST)" = "lf" ]; then \
+ $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing --lf $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ else \
+ $(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing -k "$(TEST)" $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
+ fi; \
else \
$(VENV_PYTEST) --cov=$(if $(PKG),$(PKG),pipelex) --cov-report=term-missing $(if $(filter 2,$(VERBOSE)),-vv,$(if $(filter 3,$(VERBOSE)),-vvv,-v)); \
fi
@@ -488,7 +550,7 @@ c: format lint pyright mypy
cc: cleanderived c
@echo "> done: cc = cleanderived format lint pyright pylint mypy"
-check: cc check-unused-imports pylint ccs
+check: cc check-unused-imports ccs check-rules pylint
@echo "> done: check"
v: validate
diff --git a/pipelex/cli/commands/init/backends.py b/pipelex/cli/commands/init/backends.py
index e4451ef35..b05744478 100644
--- a/pipelex/cli/commands/init/backends.py
+++ b/pipelex/cli/commands/init/backends.py
@@ -62,8 +62,12 @@ def get_selected_backend_keys(backends_toml_path: str) -> list[str]:
return selected_backends
-def customize_backends_config() -> None:
- """Interactively customize which inference backends are enabled in backends.toml."""
+def customize_backends_config(is_first_time_setup: bool = False) -> None:
+ """Interactively customize which inference backends are enabled in backends.toml.
+
+ Args:
+ is_first_time_setup: Whether this is the first time backends.toml is being set up.
+ """
console = get_console()
backends_toml_path = os.path.join(config_manager.pipelex_config_dir, "inference", "backends.toml")
template_backends_path = os.path.join(str(get_kit_configs_dir()), "inference", "backends.toml")
@@ -80,9 +84,9 @@ def customize_backends_config() -> None:
# Get currently enabled backends to show user their current selection
currently_enabled = get_currently_enabled_backends(backends_toml_path, backend_options)
- # If all backends are enabled, treat this as a fresh copy (not a meaningful selection)
- # In this case, don't use currently_enabled as the default to avoid selecting everything
- if currently_enabled and len(currently_enabled) == len(backend_options):
+ # If this is first-time setup, ignore what's in the template (all enabled)
+ # and use only pipelex_inference as the default
+ if is_first_time_setup or (currently_enabled and len(currently_enabled) == len(backend_options)):
currently_enabled = []
# Load the backends.toml file
@@ -90,8 +94,8 @@ def customize_backends_config() -> None:
console.print()
# UI: Display panel and get user selection
- console.print(build_backend_selection_panel(backend_options, currently_enabled))
- selected_indices = prompt_backend_indices(console, backend_options, currently_enabled)
+ console.print(build_backend_selection_panel(backend_options, currently_enabled, is_first_time_setup))
+ selected_indices = prompt_backend_indices(console, backend_options, currently_enabled, is_first_time_setup)
# Business logic: Update TOML
update_backends_in_toml(toml_doc, selected_indices, backend_options)
diff --git a/pipelex/cli/commands/init/command.py b/pipelex/cli/commands/init/command.py
index 8fb0addf5..cc8801f6c 100644
--- a/pipelex/cli/commands/init/command.py
+++ b/pipelex/cli/commands/init/command.py
@@ -158,6 +158,7 @@ def execute_initialization(
check_routing: bool,
backends_toml_path: str,
telemetry_config_path: str,
+ is_first_time_backends_setup: bool,
):
"""Execute the initialization steps.
@@ -172,8 +173,12 @@ def execute_initialization(
check_routing: Whether routing was in focus.
backends_toml_path: Path to backends.toml file.
telemetry_config_path: Path to telemetry config file.
+ is_first_time_backends_setup: Whether backends.toml didn't exist before this run.
"""
+ # Track if backends were just copied during config initialization
+ backends_just_copied_during_config = False
+
# Step 1: Initialize config if needed
if needs_config:
# Check if backends.toml exists before copying
@@ -184,15 +189,18 @@ def execute_initialization(
# If backends.toml was just created (freshly copied), always prompt for backend selection
backends_exists_now = path_exists(backends_toml_path)
- backends_just_copied = not backends_existed_before and backends_exists_now
+ backends_just_copied_during_config = not backends_existed_before and backends_exists_now
- if backends_just_copied or (check_inference and backends_exists_now):
+ if backends_just_copied_during_config or (check_inference and backends_exists_now):
needs_inference = True
+ # Determine if this is truly a first-time setup (either tracked from before or just copied now)
+ first_time_setup = is_first_time_backends_setup or backends_just_copied_during_config
+
# Step 2: Set up inference backends if needed
if needs_inference:
console.print()
- customize_backends_config()
+ customize_backends_config(is_first_time_setup=first_time_setup)
# Automatically set up routing after backends (unless routing is the specific focus)
if not check_routing:
@@ -253,6 +261,9 @@ def init_cmd(
check_routing = focus == InitFocus.ROUTING
check_telemetry = focus in (InitFocus.ALL, InitFocus.TELEMETRY)
+ # Track if backends.toml existed before we start
+ is_first_time_backends_setup = not path_exists(backends_toml_path)
+
# Check what needs to be initialized
needs_config, needs_inference, needs_routing, needs_telemetry = determine_needs(
reset=reset,
@@ -312,6 +323,7 @@ def init_cmd(
check_routing=check_routing,
backends_toml_path=backends_toml_path,
telemetry_config_path=telemetry_config_path,
+ is_first_time_backends_setup=is_first_time_backends_setup,
)
except typer.Exit:
diff --git a/pipelex/cli/commands/init/ui/backends_ui.py b/pipelex/cli/commands/init/ui/backends_ui.py
index c7fa48774..f75fbd989 100644
--- a/pipelex/cli/commands/init/ui/backends_ui.py
+++ b/pipelex/cli/commands/init/ui/backends_ui.py
@@ -92,12 +92,15 @@ def get_currently_enabled_backends(backends_toml_path: str, backend_options: lis
return sorted(currently_enabled)
-def build_backend_selection_panel(backend_options: list[tuple[str, str]], currently_enabled: list[int] | None = None) -> Panel:
+def build_backend_selection_panel(
+ backend_options: list[tuple[str, str]], currently_enabled: list[int] | None = None, is_first_time_setup: bool = False
+) -> Panel:
"""Create a Rich Panel for backend selection with options table.
Args:
backend_options: List of tuples (backend_key, display_name).
currently_enabled: Optional list of currently enabled backend indices (0-based).
+ is_first_time_setup: Whether this is the first time backends are being set up.
Returns:
A Panel containing the backend selection interface.
@@ -108,8 +111,8 @@ def build_backend_selection_panel(backend_options: list[tuple[str, str]], curren
table.add_column(style="dim", width=15)
for idx, (_, backend_name) in enumerate(backend_options, start=1):
- # Mark currently enabled backends
- status = "[green]✓ enabled[/green]" if currently_enabled and (idx - 1) in currently_enabled else ""
+ # Mark currently enabled backends (but not for first-time setup)
+ status = "[green]✓ enabled[/green]" if currently_enabled and (idx - 1) in currently_enabled and not is_first_time_setup else ""
table.add_row(f"[{idx}]", backend_name, status)
# Add special options at the end
@@ -117,7 +120,7 @@ def build_backend_selection_panel(backend_options: list[tuple[str, str]], curren
table.add_row("[Q]", "[dim]quit - Exit without configuring[/dim]", "")
# Update description based on whether we're showing current selection
- if currently_enabled:
+ if currently_enabled and not is_first_time_setup:
# Build current selection display with numbers and names
current_items: list[str] = []
for idx in sorted(currently_enabled):
@@ -147,13 +150,16 @@ def build_backend_selection_panel(backend_options: list[tuple[str, str]], curren
)
-def prompt_backend_indices(console: Console, backend_options: list[tuple[str, str]], currently_enabled: list[int] | None = None) -> list[int]:
+def prompt_backend_indices(
+ console: Console, backend_options: list[tuple[str, str]], currently_enabled: list[int] | None = None, is_first_time_setup: bool = False
+) -> list[int]:
"""Prompt user to select backend indices with validation.
Args:
console: Rich Console instance for user interaction.
backend_options: List of available backend options.
currently_enabled: Optional list of currently enabled backend indices (0-based).
+ is_first_time_setup: Whether this is the first time backends are being set up.
Returns:
List of validated backend indices (0-based) selected by the user.
@@ -161,11 +167,12 @@ def prompt_backend_indices(console: Console, backend_options: list[tuple[str, st
Raises:
typer.Exit: If user chooses to quit.
"""
- # Determine default based on current selection or fallback to first option
- if currently_enabled:
+ # Determine default based on current selection or fallback to first option (pipelex_inference)
+ if currently_enabled and not is_first_time_setup:
default_indices = sorted(currently_enabled)
default_str = ",".join(str(i + 1) for i in default_indices)
else:
+ # For first-time setup or no current selection, default to pipelex_inference (index 0)
default_indices = [0]
default_str = "1"
diff --git a/pipelex/cli/commands/kit_cmd.py b/pipelex/cli/commands/kit_cmd.py
index 173c8286c..ef80d7c9f 100644
--- a/pipelex/cli/commands/kit_cmd.py
+++ b/pipelex/cli/commands/kit_cmd.py
@@ -6,14 +6,51 @@ import typer
from typing_extensions import Annotated
from pipelex.cli.exceptions import PipelexCLIError
-from pipelex.kit.cursor_export import export_cursor_rules, remove_cursor_rules
+from pipelex.kit.cursor_rules import remove_cursor_rules, update_cursor_rules
from pipelex.kit.index_loader import load_index
+from pipelex.kit.index_models import KitIndex
from pipelex.kit.migrations_export import export_migration_instructions
-from pipelex.kit.targets_update import build_merged_rules, remove_from_targets, update_targets
+from pipelex.kit.single_file_agent_rules import remove_from_targets, update_single_file_agent_rules
kit_app = typer.Typer(no_args_is_help=True)
+def _sync_agent_rules(
+ repo_root: Path | None,
+ cursor: bool,
+ single_files: bool,
+ dry_run: bool,
+ diff: bool,
+ backup: str | None,
+ agent_set: str | None,
+ kit_index: KitIndex | None = None,
+) -> None:
+ resolved_repo_root = repo_root if repo_root is not None else Path()
+ loaded_kit_index = load_index() if kit_index is None else kit_index
+ agent_set = agent_set or loaded_kit_index.agent_rules.default_set
+
+ if cursor:
+ typer.echo("📤 Updating Cursor rules...")
+ update_cursor_rules(resolved_repo_root, loaded_kit_index, agent_set=agent_set, dry_run=dry_run)
+
+ if single_files:
+ typer.echo("📝 Updating target files...")
+ update_single_file_agent_rules(
+ repo_root=resolved_repo_root,
+ kit_index=loaded_kit_index,
+ agent_set=agent_set,
+ targets=loaded_kit_index.agent_rules.targets,
+ dry_run=dry_run,
+ diff=diff,
+ backup=backup,
+ )
+
+ if dry_run:
+ typer.echo("✅ Dry run completed - no changes made")
+ else:
+ typer.echo("✅ Kit sync completed successfully")
+
+
@kit_app.command("rules", help="Export Pipelex Cursor rules and merge Pipelex marked sections into other agent rules files")
def agent_rules(
repo_root: Annotated[Path | None, typer.Option("--repo-root", dir_okay=True, writable=True, help="Repository root directory")] = None,
@@ -22,27 +59,18 @@ def agent_rules(
dry_run: Annotated[bool, typer.Option("--dry-run", help="Show what would be done without making changes")] = False,
diff: Annotated[bool, typer.Option("--diff", help="Show unified diff of changes")] = False,
backup: Annotated[str | None, typer.Option("--backup", help="Backup suffix (e.g., '.bak')")] = None,
+ agent_set: Annotated[str | None, typer.Option("--set", help="Agent rule set to sync (use 'pipelex' for Pipelex repo)")] = None,
) -> None:
try:
- if repo_root is None:
- repo_root = Path()
-
- idx = load_index()
-
- if cursor:
- typer.echo("📤 Exporting Cursor rules...")
- export_cursor_rules(repo_root, idx, dry_run=dry_run)
-
- if single_files:
- typer.echo("📝 Building merged agent documentation...")
- merged_md = build_merged_rules(idx)
- typer.echo("📝 Updating target files...")
- update_targets(repo_root, merged_md, idx.agent_rules.targets, dry_run=dry_run, diff=diff, backup=backup)
-
- if dry_run:
- typer.echo("✅ Dry run completed - no changes made")
- else:
- typer.echo("✅ Kit sync completed successfully")
+ _sync_agent_rules(
+ repo_root=repo_root,
+ cursor=cursor,
+ single_files=single_files,
+ dry_run=dry_run,
+ diff=diff,
+ backup=backup,
+ agent_set=agent_set,
+ )
except Exception as exc:
msg = f"Failed to sync kit assets for agent rules: {exc}"
diff --git a/pipelex/cli/dev_cli/_dev_cli.py b/pipelex/cli/dev_cli/_dev_cli.py
index 7cf32aef7..a2c418a48 100644
--- a/pipelex/cli/dev_cli/_dev_cli.py
+++ b/pipelex/cli/dev_cli/_dev_cli.py
@@ -1,5 +1,6 @@
"""Main entry point for the internal development CLI."""
+import sys
from typing import Annotated
import typer
@@ -8,6 +9,7 @@ from typer.core import TyperGroup
from typing_extensions import override
from pipelex.cli.dev_cli.commands.check_config_sync_cmd import LeadingConfig, check_config_sync_cmd
+from pipelex.cli.dev_cli.commands.check_rules_sync_cmd import check_rules_sync_cmd
from pipelex.hub import get_console
from pipelex.tools.misc.package_utils import get_package_version
@@ -18,7 +20,7 @@ class PipelexDevCLI(TyperGroup):
@override
def list_commands(self, ctx: Context) -> list[str]:
"""List commands in proper order."""
- return ["check-config-sync"]
+ return ["check-config-sync", "check-rules"]
@override
def get_command(self, ctx: Context, cmd_name: str) -> Command | None:
@@ -46,6 +48,10 @@ app = typer.Typer(
@app.callback(invoke_without_command=True)
def app_callback(_ctx: typer.Context) -> None:
+ # Skip banner if --quiet or -q flag is present
+ if "--quiet" in sys.argv or "-q" in sys.argv:
+ return
+
console = get_console()
package_version = get_package_version()
console.print(
@@ -65,6 +71,16 @@ def check_config_sync_command(
LeadingConfig,
typer.Option(help="Which configuration is the leading (left) one: 'installed' (.pipelex) or 'kit' (pipelex/kit/configs)"),
] = LeadingConfig.KIT,
+ quiet: Annotated[bool, typer.Option("--quiet", "-q", help="Output only a single validation line")] = False,
) -> None:
"""Verify that .pipelex and pipelex/kit/configs are in sync."""
- check_config_sync_cmd(show_diff=show_diff, leading=leading)
+ check_config_sync_cmd(show_diff=show_diff, leading=leading, quiet=quiet)
+
+
[email protected](name="check-rules", help="Verify that installed agent rules match kit templates")
+def check_rules_command(
+ show_diff: Annotated[bool, typer.Option("--show-diff/--no-diff", help="Show differences if found")] = True,
+ quiet: Annotated[bool, typer.Option("--quiet", "-q", help="Output only a single validation line")] = False,
+) -> None:
+ """Verify that installed agent rules match kit templates."""
+ check_rules_sync_cmd(show_diff=show_diff, quiet=quiet)
diff --git a/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py b/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
index b43a7f70d..254911543 100644
--- a/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
+++ b/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
@@ -22,6 +22,7 @@ class LeadingConfig(StrEnum):
def check_config_sync_cmd(
show_diff: bool = True,
leading: LeadingConfig = LeadingConfig.INSTALLED,
+ quiet: bool = False,
) -> None:
"""Verify that .pipelex and pipelex/kit/configs are in sync.
@@ -30,6 +31,7 @@ def check_config_sync_cmd(
leading: Which configuration is the leading (left) one in the diff.
LeadingConfig.INSTALLED means .pipelex is the reference (default),
LeadingConfig.KIT means pipelex/kit/configs is the reference
+ quiet: If True, output only a single validation line (for use in Make targets)
"""
console = get_console()
@@ -39,15 +41,21 @@ def check_config_sync_cmd(
# Check if both directories exist
if not pipelex_dir.exists():
- console.print()
- console.print("[red]✗[/red] Directory [cyan].pipelex[/cyan] does not exist")
- console.print()
+ if quiet:
+ console.print("[red]✗ Config sync check: FAILED[/red] - .pipelex does not exist")
+ else:
+ console.print()
+ console.print("[red]✗[/red] Directory [cyan].pipelex[/cyan] does not exist")
+ console.print()
sys.exit(1)
if not configs_dir.exists():
- console.print()
- console.print("[red]✗[/red] Directory [cyan]pipelex/kit/configs[/cyan] does not exist")
- console.print()
+ if quiet:
+ console.print("[red]✗ Config sync check: FAILED[/red] - pipelex/kit/configs does not exist")
+ else:
+ console.print()
+ console.print("[red]✗[/red] Directory [cyan]pipelex/kit/configs[/cyan] does not exist")
+ console.print()
sys.exit(1)
# Determine directory order based on leading parameter
@@ -64,44 +72,48 @@ def check_config_sync_cmd(
right_label = ".pipelex"
# Check for differences
- console.print()
- console.print("[bold]Checking config synchronization...[/bold]")
- console.print(f" Leading: [cyan]{left_label}[/cyan] ↔ [cyan]{right_label}[/cyan]")
- console.print()
-
has_diff = has_diff_dirs(left_dir, right_dir)
if not has_diff:
# No differences found
- success_panel = Panel(
- "[green]✓[/green] Directories are in sync!\n\n[dim].pipelex and pipelex/kit/configs have no differences.[/dim]",
- title="[bold green]Config Sync Check: PASSED[/bold green]",
- border_style="green",
- padding=(1, 2),
- )
- console.print(success_panel)
+ if quiet:
+ console.print("[green]✓ Config sync check: PASSED[/green]")
+ else:
+ console.print()
+ console.print("[bold]Checking config synchronization...[/bold]")
+ console.print(f" Leading: [cyan]{left_label}[/cyan] ↔ [cyan]{right_label}[/cyan]")
+ console.print()
+ success_panel = Panel(
+ "[green]✓[/green] Directories are in sync!\n\n[dim].pipelex and pipelex/kit/configs have no differences.[/dim]",
+ title="[bold green]Config Sync Check: PASSED[/bold green]",
+ border_style="green",
+ padding=(1, 2),
+ )
+ console.print(success_panel)
+ console.print()
+ else:
console.print()
- sys.exit(0)
-
- # Differences found
- error_panel = Panel(
- "[red]✗[/red] Directories are [bold]NOT[/bold] in sync!\n\n[dim].pipelex and pipelex/kit/configs have differences.[/dim]",
- title="[bold red]Config Sync Check: FAILED[/bold red]",
- border_style="red",
- padding=(1, 2),
- )
- console.print(error_panel)
- console.print()
-
- if show_diff:
+ console.print("[bold]Checking config synchronization...[/bold]")
+ console.print(f" Leading: [cyan]{left_label}[/cyan] ↔ [cyan]{right_label}[/cyan]")
console.print()
- pretty_diff = make_diff_dirs_pretty(left_dir, right_dir)
- console.print(pretty_diff)
+ error_panel = Panel(
+ "[red]✗[/red] Directories are [bold]NOT[/bold] in sync!\n\n[dim].pipelex and pipelex/kit/configs have differences.[/dim]",
+ title="[bold red]Config Sync Check: FAILED[/bold red]",
+ border_style="red",
+ padding=(1, 2),
+ )
+ console.print(error_panel)
console.print()
- console.print("[bold yellow]Recommended Actions:[/bold yellow]")
- console.print(" • If [cyan].pipelex[/cyan] is correct, run: [cyan]make up-kit-configs[/cyan] or simply [cyan]make ukc[/cyan]")
- console.print(" • If [cyan]pipelex/kit/configs[/cyan] is correct, copy it to [cyan].pipelex[/cyan]")
- console.print()
+ if show_diff:
+ console.print()
+ pretty_diff = make_diff_dirs_pretty(left_dir, right_dir)
+ console.print(pretty_diff)
+ console.print()
- sys.exit(1)
+ console.print("[bold yellow]Recommended Actions:[/bold yellow]")
+ console.print(" • If [cyan].pipelex[/cyan] is correct, run: [cyan]make up-kit-configs[/cyan] or simply [cyan]make ukc[/cyan]")
+ console.print(" • If [cyan]pipelex/kit/configs[/cyan] is correct, copy it to [cyan].pipelex[/cyan]")
+ console.print()
+
+ sys.exit(1)
diff --git a/pipelex/cli/dev_cli/commands/check_rules_sync_cmd.py b/pipelex/cli/dev_cli/commands/check_rules_sync_cmd.py
new file mode 100644
index 000000000..0d068a81b
--- /dev/null
+++ b/pipelex/cli/dev_cli/commands/check_rules_sync_cmd.py
@@ -0,0 +1,127 @@
+"""Command to verify installed agent rules match kit templates."""
+
+from __future__ import annotations
+
+import re
+import sys
+from pathlib import Path
+
+from rich.panel import Panel
+
+from pipelex.hub import get_console
+from pipelex.kit.index_loader import load_index
+from pipelex.kit.markers import find_span, replace_span, wrap
+from pipelex.kit.single_file_agent_rules import build_merged_rules, insert_block_with_markers, unified_diff
+
+
+def check_rules_sync_cmd(show_diff: bool = True, quiet: bool = False) -> None:
+ """Verify that installed agent rules match kit templates.
+
+ Args:
+ show_diff: If True, display the differences when found
+ quiet: If True, output only a single validation line (for use in Make targets)
+ """
+ console = get_console()
+ kit_index = load_index()
+ agent_set = "coding_standards"
+
+ missing_targets: list[Path] = []
+ mismatches: list[tuple[Path, str, str]] = []
+
+ for target in kit_index.agent_rules.targets.values():
+ # Check if this target has its own set override for the agent_set
+ target_file_list: list[str] | None = None
+ if target.sets and agent_set in target.sets:
+ target_file_list = target.sets[agent_set]
+
+ # Build merged rules for this specific target (with override if applicable)
+ merged_rules = build_merged_rules(kit_index, agent_set=agent_set, file_list=target_file_list)
+
+ target_path = Path(target.path)
+
+ if not target_path.exists():
+ missing_targets.append(target_path)
+ continue
+
+ current_content = target_path.read_text(encoding="utf-8")
+ span = find_span(current_content, target.marker_begin, target.marker_end)
+
+ if span:
+ before_markers = current_content[: span[0]]
+ after_markers = current_content[span[1] :]
+ outside_content = before_markers + after_markers
+
+ has_h1_outside = bool(re.search(r"^#\s+.+$", outside_content, flags=re.MULTILINE))
+
+ if target.heading_1 and not has_h1_outside:
+ content_with_heading = f"{target.heading_1}\n\n{merged_rules}"
+ wrapped_block = wrap(target.marker_begin, target.marker_end, content_with_heading)
+ else:
+ wrapped_block = wrap(target.marker_begin, target.marker_end, merged_rules)
+
+ expected_content = replace_span(current_content, span, wrapped_block)
+ else:
+ expected_content = insert_block_with_markers(
+ current_content,
+ merged_rules,
+ target.heading_1,
+ (target.marker_begin, target.marker_end),
+ )
+
+ if current_content != expected_content:
+ mismatches.append((target_path, current_content, expected_content))
+
+ if not missing_targets and not mismatches:
+ if quiet:
+ console.print("[green]✓ Agent rules sync check: PASSED[/green]")
+ else:
+ console.print()
+ console.print("[bold]Checking agent rules synchronization...[/bold]")
+ console.print(" Set: [cyan]coding_standards[/cyan]")
+ console.print()
+ success_panel = Panel(
+ "[green]✓[/green] Agent rules are in sync!\n\n[dim]Installed targets match templates from pipelex/kit/agent_rules.[/dim]",
+ title="[bold green]Agent Rules Sync Check: PASSED[/bold green]",
+ border_style="green",
+ padding=(1, 2),
+ )
+ console.print(success_panel)
+ console.print()
+ else:
+ console.print()
+ console.print("[bold]Checking agent rules synchronization...[/bold]")
+ console.print(" Set: [cyan]coding_standards[/cyan]")
+ console.print()
+ error_panel = Panel(
+ (
+ "[red]✗[/red] Agent rules are [bold]NOT[/bold] in sync!\n\n"
+ "[dim]Installed targets differ from templates in pipelex/kit/agent_rules.[/dim]"
+ ),
+ title="[bold red]Agent Rules Sync Check: FAILED[/bold red]",
+ border_style="red",
+ padding=(1, 2),
+ )
+ console.print(error_panel)
+ console.print()
+
+ if missing_targets:
+ console.print("[bold yellow]Missing targets:[/bold yellow]")
+ for missing_path in missing_targets:
+ console.print(f" • [cyan]{missing_path}[/cyan]")
+ console.print()
+
+ if mismatches:
+ console.print("[bold yellow]Mismatched targets:[/bold yellow]")
+ for mismatched_path, current_content, expected_content in mismatches:
+ console.print(f" • [cyan]{mismatched_path}[/cyan]")
+ if show_diff:
+ diff_output = unified_diff(current_content, expected_content, str(mismatched_path))
+ if diff_output:
+ console.print()
+ console.print(diff_output)
+ console.print()
+
+ console.print("[bold yellow]Recommended Actions:[/bold yellow]")
+ console.print(" • Run [cyan]make rules[/cyan] to sync agent rules")
+ console.print()
+ sys.exit(1)
diff --git a/pipelex/kit/agent_rules/codex_commands.md b/pipelex/kit/agent_rules/codex_commands.md
new file mode 100644
index 000000000..244eb3592
--- /dev/null
+++ b/pipelex/kit/agent_rules/codex_commands.md
@@ -0,0 +1,62 @@
+# Codex Cloud Commands
+
+## Linting
+
+ After making code changes, you must always lint using `make check`.
+
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
+
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
+
+ Always fix any issues reported by these tools before proceeding.
+
+## Running Tests in Codex Cloud
+
+ To test everything that can be tested from within the Codex Cloud sandbox, run this:
+
+ ```bash
+ make codex-tests
+ # It's equivalent to running pytest with `-m "(dry_runnable or not inference) and not (pipelex_api or codex_disabled)"`
+ # If some test fails, re-run it with `-s -vv` to see more details
+ ```
+
+---
+
+## Prerequisites for running command lines: activate virtual environment
+
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
+
+ Do this:
+
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
+
+ or do that:
+
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
+
+ (adapt the above command to the OS and available virtual environment name)
+
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
+
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
+
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
+
diff --git a/pipelex/kit/agent_rules/commands.md b/pipelex/kit/agent_rules/commands.md
index 83ef8cd48..996c24265 100644
--- a/pipelex/kit/agent_rules/commands.md
+++ b/pipelex/kit/agent_rules/commands.md
@@ -1,4 +1,91 @@
# Commands
- - When you want to run commands such as `python`, `pytest` or any of our CLI such as `pipelex` or `cocode`, ALWAYS use the current obvious virtual env. If the installaton is standard, the venv is named `.venv` so always check that first.
-
\ No newline at end of file
+## Linting
+
+ After making code changes, you must always lint using `make check`.
+
+ ```bash
+ make check
+ # If the current system doesn't have the `make` command, lookup the "check" target in the Makefile and run the command manually.
+ ```
+
+ This runs multiple code quality tools:
+ - Pyright: Static type checking
+ - Ruff: Fast Python linter
+ - Mypy: Static type checker
+
+ Always fix any issues reported by these tools before proceeding.
+
+## Running Tests
+
+ After you're finished making code changes, you must always run tests using `make test-xdist`.
+
+ ```bash
+ make test-xdist
+ # If the current system doesn't have the `make` command, lookup the "test-xdist" target in the Makefile and run the command manually.
+ # If some test failes, re-run it with `-s -vv` to see more details
+ ```
+
+## Running Tests with Prints
+
+ If anything went wrong, you can run the tests with prints to see the error:
+
+ ```bash
+ make test-with-prints
+ # If the current system doesn't have the `make` command, lookup the "test-with-prints" target in the Makefile and run the command manually.
+ ```
+
+## Running specific Tests
+
+ ```bash
+ make tp TEST=TestClassName
+ # or
+ make tp TEST=test_function_name
+ ```
+ Note: Matches names starting with the provided string.
+
+## Running Last Failed Tests
+
+ To rerun only the tests that failed in the previous run, use:
+
+ ```bash
+ make tp TEST=LF
+ # or with any test target
+ make test TEST=LF
+ make t TEST=LF
+ ```
+ Note: `TEST=LF` (or `TEST=lf`) will use pytest's `--lf` flag instead of name filtering.
+
+---
+
+## Prerequisites for running command lines: activate virtual environment
+
+ **CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. The only exceptions are our `make` commands which already include the env activation.
+
+ Do this:
+
+ ```bash
+ source .venv/bin/activate
+ pytest -s -v -k test_render_jinja2_from_text
+ pipelex validate all
+ ```
+
+ or do that:
+
+ ```bash
+ .venv/bin/python -m pytest -s -v -k test_render_jinja2_from_text
+ .venv/bin/pipelex validate all
+ ```
+
+ (adapt the above command to the OS and available virtual environment name)
+
+ For standard installations, the virtual environment is named `.venv`. Always check this first:
+
+ ```bash
+ # Activate the virtual environment (standard installation)
+ source .venv/bin/activate # On macOS/Linux
+ # or
+ .venv\Scripts\activate # On Windows
+ ```
+
+ If the installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
diff --git a/pipelex/kit/agent_rules/docs.md b/pipelex/kit/agent_rules/docs.md
index 67c5674b2..440d5c534 100644
--- a/pipelex/kit/agent_rules/docs.md
+++ b/pipelex/kit/agent_rules/docs.md
@@ -1,7 +1,5 @@
# Writing Docs
-Write docs and answer questions about writing docs.
-
We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.
## MkDocs Markdown Requirements
diff --git a/pipelex/kit/agent_rules/llms.md b/pipelex/kit/agent_rules/llms.md
index e8b5979ed..fbe088d9c 100644
--- a/pipelex/kit/agent_rules/llms.md
+++ b/pipelex/kit/agent_rules/llms.md
@@ -49,7 +49,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp
Examples:
```toml
llm_to_engineer = { model = "base-claude", temperature = 1 }
-llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }
+llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" }
```
The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:
diff --git a/pipelex/kit/agent_rules/pytest_standards.md b/pipelex/kit/agent_rules/pytest_standards.md
index 8e06a0db8..230a039c0 100644
--- a/pipelex/kit/agent_rules/pytest_standards.md
+++ b/pipelex/kit/agent_rules/pytest_standards.md
@@ -1,8 +1,8 @@
-# Writing unit tests
+# Writing tests
## Unit test generalities
-NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
+NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
### Test file structure
@@ -13,9 +13,8 @@ NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
-- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `tests.pipelex.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
-- Always put test inside Test classes.
-- The pipelex pipelines should be stored in `tests/test_pipelines` as well as the related structured Output classes that inherit from `StructuredContent`
+- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
+- Always put test inside Test classes: 1 TestClass per module.
### Markers
@@ -57,45 +56,23 @@ class TestFooBar:
# Test implementation
```
-- Never more than 1 class per test file.
+- Never more than 1 class per test module.
- When testing one method, if possible, limit the number of test functions, but with different test cases in parameters
- Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.
### Test Data Organization
- If it's not already there, create a `test_data.py` file in the proper test directory
-- Define test cases using `StuffBlueprint`:
-
-```python
-class TestCases:
- CASE_BLUEPRINT_1 = StuffBlueprint(
- name="test_case_1",
- concept_code="domain.ConceptName1",
- value="test_value"
- )
- CASE_BLUEPRINT_2 = StuffBlueprint(
- name="test_case_2",
- concept_code="domain.ConceptName2",
- value="test_value"
- )
-
- CASE_BLUEPRINTS: ClassVar[list[tuple[str, str]]] = [ # topic, blueprint"
- ("topic1", CASE_BLUEPRINT_1),
- ("topic2", CASE_BLUEPRINT_2),
- ]
-```
-
-Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
-Also note that we provide a topic for the test case, which is purely for convenience.
+- Note how we avoid initializing a default mutable value within a class instance, instead we use ClassVar.
+- Also note that we provide a topic for the test case, which is purely for convenience.
## Best Practices for Testing
-- Whenever possible, use strong asserts to test value, not just type and presence.
+- Use strong asserts: test value, not just type and presence.
- Use parametrize for multiple test cases
- Test both success and failure cases
- Verify working memory state
- Check output structure and content
- Use meaningful test case names
-- Include docstrings explaining test purpose but not on top of the file
+- Include docstrings explaining test purpose but not on top of the file and not on top of the class.
- Log outputs for debugging
-- Generate reports for cost tracking
diff --git a/pipelex/kit/agent_rules/python_standards.md b/pipelex/kit/agent_rules/python_standards.md
index 06db00371..eba03917d 100644
--- a/pipelex/kit/agent_rules/python_standards.md
+++ b/pipelex/kit/agent_rules/python_standards.md
@@ -14,6 +14,10 @@ This document outlines the core coding standards, best practices, and quality co
- When defining enums related to string values, always inherit from `StrEnum`
- Never test equality to an enum value: use match/case, even to single out 1 case out of 10 cases. To avoid heavy match/case code in awkward places, add methods to the enum class such as `is_foobar()`. This is to avoid bugs: when new enum values are added we want the linter to complain. Use the `|` operator to group cases
- As our match/case constructs over enums are always exhaustive, NEVER add a default `case _: ...`. Otherwise, you won't pass linting.
+ - `StrEnum` must be imported from `pipelex.types` (handles python retrocompatibility):
+ ```python
+ from pipelex.types import StrEnum
+ ```
## Imports
@@ -76,17 +80,17 @@ This document outlines the core coding standards, best practices, and quality co
## Factory Pattern
- - Use Factory Pattern for object creation when dealing with multiple implementations
+ - Use factory pattern for object creation when dealing with multiple implementations
- Our factory methods are named `make_from_...` and such
## Error Handling
- Always catch exceptions at the place where you can add useful context to it.
- Use try/except blocks with specific exceptions
- - Convert third-party exceptions to our custom ones
+ - Convert third-party exceptions to our custom ones except in pydantic validators where you can raise a ValueError or a TypeError
- NEVER catch the generic Exception, only catch specific exceptions, except at the root of CLI commands
- - NEVER raise generic exceptions like ValueError or TypeError, create new error classes and raise them instead
- - Always add `from exc` to the exception
+ - Always add `from exc` to the exception raise statements
+ - Always write the error message as a variable before raising it, for cleaner error traces
```python
try:
@@ -96,8 +100,6 @@ This document outlines the core coding standards, best practices, and quality co
raise PipelexSetupError(msg) from exc
```
- **Note**: Following Ruff rules, we set the error message as a variable before raising it, for cleaner error traces.
-
## Documentation
1. **Docstring Format**
@@ -123,48 +125,3 @@ This document outlines the core coding standards, best practices, and quality co
Provides methods for resizing, converting, and optimizing images.
"""
```
-
-## Code Quality Checks
-
-### Linting and Type Checking
-
-Before finalizing a task, run:
-```bash
-make fix-unused-imports
-make check
-```
-
-This runs multiple code quality tools:
-- Pyright: Static type checking
-- Ruff: Fast Python linter
-- Mypy: Static type checker
-
-Always fix any issues reported by these tools before proceeding.
-
-### Running Tests
-
-1. **Quick Test Run** (no LLM/image generation):
- ```bash
- make tp
- ```
- Runs tests with markers: `(dry_runnable or not (inference or llm or img_gen or extract)) and not (needs_output or pipelex_api)`
-
-2. **Specific Tests**:
- ```bash
- make tp TEST=TestClassName
- # or
- make tp TEST=test_function_name
- ```
- Note: Matches names starting with the provided string.
-
-**Important**: Never run `make ti`, `make test-inference`, `make te`, `make test-extract`, `make tg`, or `make test-img-gen` - these use costly inference.
-
-## Pipelines
-
-- Always validate pipelines after creation/edit with `make validate`.
- Iterate if there are errors.
-
-## Project Structure
-
-- **Tests**: `tests/` directory
-- **Documentation**: `docs/` directory
diff --git a/pipelex/kit/agent_rules/run_pipelex.md b/pipelex/kit/agent_rules/run_pipelex.md
index 42d696587..2f146d388 100644
--- a/pipelex/kit/agent_rules/run_pipelex.md
+++ b/pipelex/kit/agent_rules/run_pipelex.md
@@ -1,20 +1,5 @@
# Guide to execute a pipeline and write example code
-## Prerequisites: Virtual Environment
-
-**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.
-
-For standard installations, the virtual environment is named `.venv`. Always check this first:
-
-```bash
-# Activate the virtual environment (standard installation)
-source .venv/bin/activate # On macOS/Linux
-# or
-.venv\Scripts\activate # On Windows
-```
-
-If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.
-
## Example to execute a pipeline with text output
```python
diff --git a/pipelex/kit/agent_rules/tdd.md b/pipelex/kit/agent_rules/tdd.md
index 0f41c71e8..a7800b1bf 100644
--- a/pipelex/kit/agent_rules/tdd.md
+++ b/pipelex/kit/agent_rules/tdd.md
@@ -5,7 +5,6 @@ This document outlines our test-driven development (TDD) process and the tools a
## TDD Cycle
1. **Write a Test First**
-[pytest.mdc](pytest.mdc)
2. **Write the Code**
- Implement the minimum amount of code needed to pass the test
@@ -13,12 +12,8 @@ This document outlines our test-driven development (TDD) process and the tools a
- Keep it simple - don't write more than needed
3. **Run Linting and Type Checking**
-[coding_standards.mdc](coding_standards.mdc)
-4. **Refactor if needed**
-If the code needs refactoring, with the best practices [coding_standards.mdc](coding_standards.mdc)
+4. **Validate tests**
-5. **Validate tests**
-
-Remember: The key to TDD is writing the test first and letting it drive your implementation. Always run the full test suite and quality checks before considering a feature complete.
+Remember: The key to TDD is writing the test first and letting it drive your implementation. Then, always run the full test suite and quality checks before considering a feature complete.
diff --git a/pipelex/kit/configs/inference/backends.toml b/pipelex/kit/configs/inference/backends.toml
index 52fe3cbfc..2f002afc8 100644
--- a/pipelex/kit/configs/inference/backends.toml
+++ b/pipelex/kit/configs/inference/backends.toml
@@ -71,7 +71,7 @@ api_key = "${OPENAI_API_KEY}"
[vertexai]
display_name = "Google Vertex AI"
-enabled = true
+enabled = false # This is the only one we disable beacuse setting it up requires internet access just to get credentials so it fails in CI sandboxes
gcp_project_id = "${GCP_PROJECT_ID}"
gcp_location = "${GCP_LOCATION}"
gcp_credentials_file_path = "${GCP_CREDENTIALS_FILE_PATH}"
diff --git a/pipelex/kit/configs/inference/backends/azure_openai.toml b/pipelex/kit/configs/inference/backends/azure_openai.toml
index f18fb2371..2062251bb 100644
--- a/pipelex/kit/configs/inference/backends/azure_openai.toml
+++ b/pipelex/kit/configs/inference/backends/azure_openai.toml
@@ -97,24 +97,28 @@ model_id = "gpt-5-mini-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 0.25, output = 2.0 }
+constraints = ["temperature_must_be_1"]
[gpt-5-nano]
model_id = "gpt-5-nano-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 0.05, output = 0.4 }
+constraints = ["temperature_must_be_1"]
[gpt-5-chat]
model_id = "gpt-5-chat-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 1.25, output = 10.0 }
+constraints = ["temperature_must_be_1"]
[gpt-5]
model_id = "gpt-5-2025-08-07"
inputs = ["text", "images"]
outputs = ["text", "structured"]
costs = { input = 1.25, output = 10.0 }
+constraints = ["temperature_must_be_1"]
################################################################################
# IMAGE GENERATION MODELS
diff --git a/pipelex/kit/configs/inference/routing_profiles.toml b/pipelex/kit/configs/inference/routing_profiles.toml
index fa3f1d42a..0910ff285 100644
--- a/pipelex/kit/configs/inference/routing_profiles.toml
+++ b/pipelex/kit/configs/inference/routing_profiles.toml
@@ -94,10 +94,6 @@ default = "openai"
description = "Use Pipelex Inference backend for all its supported models"
default = "pipelex_inference"
-[profiles.all_vertexai]
-description = "Use Vertex AI backend for all its supported models"
-default = "vertexai"
-
[profiles.all_xai]
description = "Use xAI backend for all its supported models"
default = "xai"
diff --git a/pipelex/kit/cursor_export.py b/pipelex/kit/cursor_rules.py
similarity index 64%
rename from pipelex/kit/cursor_export.py
rename to pipelex/kit/cursor_rules.py
index ebc8b38d8..a89082fc7 100644
--- a/pipelex/kit/cursor_export.py
+++ b/pipelex/kit/cursor_rules.py
@@ -6,6 +6,7 @@ from typing import Any
import typer
import yaml
+from pipelex.kit.exceptions import KitError
from pipelex.kit.index_models import KitIndex
from pipelex.kit.paths import get_kit_agents_dir
@@ -24,40 +25,50 @@ def _iter_agent_files(agents_dir: Traversable) -> Iterable[tuple[str, str]]:
yield child.name, child.read_text(encoding="utf-8")
-def _front_matter_for(name: str, idx: KitIndex) -> dict[str, Any]:
+def _front_matter_for(name: str, kit_index: KitIndex) -> dict[str, Any]:
"""Build front-matter for a specific file.
Args:
name: Filename (e.g., "pytest_standards.md")
- idx: Kit index configuration
+ kit_index: Kit index configuration
Returns:
Merged front-matter dictionary
"""
- base = idx.agent_rules.cursor.front_matter.copy()
+ base = kit_index.agent_rules.cursor.front_matter.copy()
key = name.removesuffix(".md")
- if key in idx.agent_rules.cursor.files:
- base |= idx.agent_rules.cursor.files[key].front_matter
+ if key in kit_index.agent_rules.cursor.files:
+ base |= kit_index.agent_rules.cursor.files[key].front_matter
# Remove globs if it's an empty list
if "globs" in base and base["globs"] == []:
del base["globs"]
return base
-def export_cursor_rules(repo_root: Path, idx: KitIndex, dry_run: bool = False) -> None:
- """Export agent markdown files to Cursor .mdc files with YAML front-matter.
+def update_cursor_rules(repo_root: Path, kit_index: KitIndex, agent_set: str, dry_run: bool = False) -> None:
+ """Update Cursor .mdc rule files from agent markdown with YAML front-matter.
Args:
repo_root: Repository root directory
- idx: Kit index configuration
+ kit_index: Kit index configuration
+ agent_set: Agent set to limit exported files
dry_run: If True, only print what would be done
"""
agents_dir = get_kit_agents_dir()
out_dir = repo_root / ".cursor" / "rules"
out_dir.mkdir(parents=True, exist_ok=True)
+ allowed_files: set[str] | None = None
+ rules_set = kit_index.agent_rules.sets.get(agent_set)
+ if not rules_set:
+ msg = f"Agent set '{agent_set}' not found in index.toml. Available sets: {list(kit_index.agent_rules.sets.keys())}"
+ raise KitError(msg)
+ allowed_files = set(rules_set)
+
for fname, body in _iter_agent_files(agents_dir):
- fm = _front_matter_for(fname, idx)
+ if fname not in allowed_files:
+ continue
+ fm = _front_matter_for(fname, kit_index)
yaml_block = "---\n" + yaml.safe_dump(fm, sort_keys=False).rstrip() + "\n---\n"
mdc = yaml_block + body
out_path = out_dir / (fname.removesuffix(".md") + ".mdc")
@@ -65,8 +76,13 @@ def export_cursor_rules(repo_root: Path, idx: KitIndex, dry_run: bool = False) -
if dry_run:
typer.echo(f"[DRY] write {out_path}")
else:
- out_path.write_text(mdc, encoding="utf-8")
- typer.echo(f"✅ Exported {out_path}")
+ # Only write and print if content changed
+ existing_content = out_path.read_text(encoding="utf-8") if out_path.exists() else None
+ if existing_content != mdc:
+ out_path.write_text(mdc, encoding="utf-8")
+ typer.echo(f"✅ Exported {out_path}")
+ else:
+ typer.echo(f"⚪ Unchanged {out_path}")
def remove_cursor_rules(repo_root: Path, dry_run: bool = False) -> None:
diff --git a/pipelex/kit/exceptions.py b/pipelex/kit/exceptions.py
new file mode 100644
index 000000000..5f11b96f8
--- /dev/null
+++ b/pipelex/kit/exceptions.py
@@ -0,0 +1,5 @@
+from pipelex.base_exceptions import PipelexError
+
+
+class KitError(PipelexError):
+ pass
diff --git a/pipelex/kit/index.toml b/pipelex/kit/index.toml
index b05cd6e95..5c420bc28 100644
--- a/pipelex/kit/index.toml
+++ b/pipelex/kit/index.toml
@@ -65,28 +65,37 @@ globs = ["*.plx", "*.toml"]
path = "AGENTS.md"
marker_begin = "<!-- BEGIN_PIPELEX_RULES -->"
marker_end = "<!-- END_PIPELEX_RULES -->"
-heading_1 = "# Pipelex Rules"
+heading_1 = "# Pipelex Coding Rules"
+
+[agent_rules.targets.agents.sets]
+coding_standards = [
+ "codex_commands.md",
+ "python_standards.md",
+ "pytest_standards.md",
+ "docs.md",
+ "tdd.md",
+]
[agent_rules.targets.claude]
path = "CLAUDE.md"
marker_begin = "<!-- BEGIN_PIPELEX_RULES -->"
marker_end = "<!-- END_PIPELEX_RULES -->"
-heading_1 = "# Pipelex Rules"
+heading_1 = "# Pipelex Coding Rules"
[agent_rules.targets.github_copilot]
path = ".github/copilot-instructions.md"
marker_begin = "<!-- BEGIN_PIPELEX_RULES -->"
marker_end = "<!-- END_PIPELEX_RULES -->"
-heading_1 = "# Pipelex Rules"
+heading_1 = "# Pipelex Coding Rules"
[agent_rules.targets.windsurf]
path = ".windsurfrules.md"
marker_begin = "<!-- BEGIN_PIPELEX_RULES -->"
marker_end = "<!-- END_PIPELEX_RULES -->"
-heading_1 = "# Pipelex Rules"
+heading_1 = "# Pipelex Coding Rules"
[agent_rules.targets.blackbox]
path = ".blackboxrules"
marker_begin = "<!-- BEGIN_PIPELEX_RULES -->"
marker_end = "<!-- END_PIPELEX_RULES -->"
-heading_1 = "# Pipelex Rules"
+heading_1 = "# Pipelex Coding Rules"
diff --git a/pipelex/kit/index_models.py b/pipelex/kit/index_models.py
index 54e3ffd2f..8c32ac30b 100644
--- a/pipelex/kit/index_models.py
+++ b/pipelex/kit/index_models.py
@@ -25,6 +25,9 @@ class Target(ConfigModel):
marker_begin: str = Field(description="Beginning marker for content insertion")
marker_end: str = Field(description="Ending marker for content insertion")
heading_1: str | None = Field(default=None, description="Main title (H1) to add when inserting into empty file or file with no H1 headings")
+ sets: dict[str, list[str]] | None = Field(
+ default=None, description="Target-specific named sets overrides for agent_rules files (e.g., coding_standards, pipelex_language, all)"
+ )
class AgentRules(ConfigModel):
diff --git a/pipelex/kit/targets_update.py b/pipelex/kit/single_file_agent_rules.py
similarity index 72%
rename from pipelex/kit/targets_update.py
rename to pipelex/kit/single_file_agent_rules.py
index c4038b81f..862185485 100644
--- a/pipelex/kit/targets_update.py
+++ b/pipelex/kit/single_file_agent_rules.py
@@ -47,37 +47,44 @@ def _demote_headings(md_content: str, levels: int) -> str:
return re.sub(pattern, demote_match, md_content, flags=re.MULTILINE)
-# TODO: fix bug which makes it not idempotent (because heading 1 gets deleted)
-def build_merged_rules(idx: KitIndex, agent_set: str | None = None) -> str:
+def build_merged_rules(kit_index: KitIndex, agent_set: str | None = None, file_list: list[str] | None = None) -> str:
"""Build merged agent documentation from ordered files.
Args:
- idx: Kit index configuration
- agent_set: Name of the agent set to use (defaults to idx.agents.default_set)
+ kit_index: Kit index configuration
+ agent_set: Name of the agent set to use (defaults to kit_index.agents.default_set)
+ file_list: Optional explicit list of files to merge (overrides agent_set lookup)
Returns:
Merged markdown content with demoted headings
"""
agents_dir = get_kit_agents_dir()
- if agent_set is None:
- agent_set = idx.agent_rules.default_set
+ # If explicit file list provided, use it
+ if file_list is not None:
+ files_to_merge = file_list
+ else:
+ # Otherwise, look up the agent set
+ if agent_set is None:
+ agent_set = kit_index.agent_rules.default_set
+
+ if agent_set not in kit_index.agent_rules.sets:
+ msg = f"Agent set '{agent_set}' not found in index.toml. Available sets: {list(kit_index.agent_rules.sets.keys())}"
+ raise ValueError(msg)
- if agent_set not in idx.agent_rules.sets:
- msg = f"Agent set '{agent_set}' not found in index.toml. Available sets: {list(idx.agent_rules.sets.keys())}"
- raise ValueError(msg)
+ files_to_merge = kit_index.agent_rules.sets[agent_set]
parts: list[str] = []
- for name in idx.agent_rules.sets[agent_set]:
+ for name in files_to_merge:
md = _read_agent_file(agents_dir, name)
- demoted = _demote_headings(md, idx.agent_rules.demote)
+ demoted = _demote_headings(md, kit_index.agent_rules.demote)
parts.append(demoted.rstrip())
return ("\n\n".join(parts)).strip() + "\n"
-def _insert_block_with_markers(target_md: str, block_md: str, main_title: str | None, markers: tuple[str, str]) -> str:
+def insert_block_with_markers(target_md: str, block_md: str, main_title: str | None, markers: tuple[str, str]) -> str:
"""Insert block into target markdown using marker-based logic.
Args:
@@ -110,7 +117,7 @@ def _insert_block_with_markers(target_md: str, block_md: str, main_title: str |
return target_md.rstrip() + "\n\n" + wrapped_block + "\n"
-def _diff(before: str, after: str, path: str) -> str:
+def unified_diff(before: str, after: str, path: str) -> str:
"""Generate unified diff between before and after.
Args:
@@ -131,37 +138,61 @@ def _diff(before: str, after: str, path: str) -> str:
)
-def update_targets(
+def update_single_file_agent_rules(
repo_root: Path,
- merged_rules: str,
+ kit_index: KitIndex,
+ agent_set: str,
targets: dict[str, Target],
dry_run: bool,
diff: bool,
backup: str | None,
) -> None:
- """Update target files with merged agent documentation.
+ """Update single-file agent rules targets with merged agent documentation.
Args:
repo_root: Repository root directory
- merged_rules: Merged markdown content to insert
+ kit_index: Kit index configuration
+ agent_set: Default agent set to use (can be overridden by target.sets)
targets: Dictionary of target file configurations keyed by ID
dry_run: If True, only print what would be done
diff: If True, show unified diff
backup: Backup suffix (e.g., ".bak"), or None for no backup
"""
for target in targets.values():
+ # Check if this target has its own set override for the given agent_set
+ target_file_list: list[str] | None = None
+ if target.sets and agent_set in target.sets:
+ target_file_list = target.sets[agent_set]
+
+ # Build merged rules for this specific target (with override if applicable)
+ merged_rules = build_merged_rules(kit_index, agent_set=agent_set, file_list=target_file_list)
+
target_path = repo_root / target.path
before = target_path.read_text(encoding="utf-8") if target_path.exists() else ""
span = find_span(before, target.marker_begin, target.marker_end)
if span:
- # Markers exist - replace content between them
- wrapped_block = wrap(target.marker_begin, target.marker_end, merged_rules)
+ # Markers exist - need to check if we should preserve heading_1
+ # Check if the entire file (outside markers) has an H1 heading
+ before_markers = before[: span[0]]
+ after_markers = before[span[1] :]
+ outside_content = before_markers + after_markers
+
+ h1_pattern = r"^#\s+.+$"
+ has_h1_outside = bool(re.search(h1_pattern, outside_content, flags=re.MULTILINE))
+
+ # Add heading_1 if it's configured and there's no H1 outside markers
+ if target.heading_1 and not has_h1_outside:
+ content_with_heading = f"{target.heading_1}\n\n{merged_rules}"
+ wrapped_block = wrap(target.marker_begin, target.marker_end, content_with_heading)
+ else:
+ wrapped_block = wrap(target.marker_begin, target.marker_end, merged_rules)
+
after = replace_span(before, span, wrapped_block)
else:
# No markers - insert with markers
- after = _insert_block_with_markers(
+ after = insert_block_with_markers(
before,
merged_rules,
target.heading_1,
@@ -171,10 +202,11 @@ def update_targets(
if dry_run:
typer.echo(f"[DRY] update {target_path}")
if diff:
- diff_output = _diff(before, after, str(target_path))
+ diff_output = unified_diff(before, after, str(target_path))
if diff_output:
typer.echo(diff_output)
- else:
+ # Only write and print if content changed
+ elif before != after:
if backup and target_path.exists():
backup_path = target_path.with_suffix(target_path.suffix + backup)
backup_path.write_text(before, encoding="utf-8")
@@ -185,9 +217,11 @@ def update_targets(
typer.echo(f"✅ Updated {target_path}")
if diff:
- diff_output = _diff(before, after, str(target_path))
+ diff_output = unified_diff(before, after, str(target_path))
if diff_output:
typer.echo(diff_output)
+ else:
+ typer.echo(f"⚪ Unchanged {target_path}")
def remove_from_targets(
@@ -264,7 +298,7 @@ def remove_from_targets(
if dry_run:
typer.echo(f"[DRY] remove marked section from {target_path}")
if diff:
- diff_output = _diff(before, after, str(target_path))
+ diff_output = unified_diff(before, after, str(target_path))
if diff_output:
typer.echo(diff_output)
else:
@@ -277,6 +311,6 @@ def remove_from_targets(
typer.echo(f"✅ Removed marked section from {target_path}")
if diff:
- diff_output = _diff(before, after, str(target_path))
+ diff_output = unified_diff(before, after, str(target_path))
if diff_output:
typer.echo(diff_output)
diff --git a/pipelex/pipelex.py b/pipelex/pipelex.py
index bbcadfee5..0f736eddb 100644
--- a/pipelex/pipelex.py
+++ b/pipelex/pipelex.py
@@ -48,7 +48,7 @@ from pipelex.reporting.reporting_protocol import ReportingNoOp, ReportingProtoco
from pipelex.system.configuration.config_loader import config_manager
from pipelex.system.configuration.config_root import ConfigRoot
from pipelex.system.configuration.configs import ConfigPaths, PipelexConfig
-from pipelex.system.environment import get_optional_env
+from pipelex.system.environment import is_env_var_truthy
from pipelex.system.registries.func_registry import func_registry
from pipelex.system.runtime import IntegrationMode, runtime_manager
from pipelex.system.telemetry.observer_telemetry import ObserverTelemetry
@@ -269,9 +269,9 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.
self.telemetry_manager = TelemetryManagerNoOp()
log.debug("Telemetry is disabled because telemetry_mode is set to 'off'")
case TelemetryMode.ANONYMOUS | TelemetryMode.IDENTIFIED:
- if telemetry_config.respect_dnt and (dnt := get_optional_env(DO_NOT_TRACK_ENV_VAR_KEY)) and dnt.lower() not in ["false", "0"]:
+ if telemetry_config.respect_dnt and is_env_var_truthy(DO_NOT_TRACK_ENV_VAR_KEY):
self.telemetry_manager = TelemetryManagerNoOp()
- log.debug(f"Telemetry is disabled by env var 'DO_NOT_TRACK' which is set to {dnt}")
+ log.debug(f"Telemetry is disabled by env var '{DO_NOT_TRACK_ENV_VAR_KEY}'")
else:
self.telemetry_manager = telemetry_manager or TelemetryManager(telemetry_config=telemetry_config)
else:
diff --git a/pipelex/system/configuration/config_loader.py b/pipelex/system/configuration/config_loader.py
index 1b8a416a3..2d55dc22f 100644
--- a/pipelex/system/configuration/config_loader.py
+++ b/pipelex/system/configuration/config_loader.py
@@ -6,7 +6,7 @@ from pipelex.system.configuration.config_root import (
CONFIG_BASE_OVERRIDES_AFTER_ENV,
CONFIG_BASE_OVERRIDES_BEFORE_ENV,
)
-from pipelex.system.runtime import RunMode, runtime_manager
+from pipelex.system.runtime import runtime_manager
from pipelex.tools.misc.json_utils import deep_update
from pipelex.tools.misc.toml_utils import load_toml_from_path, load_toml_from_path_if_exists
@@ -104,12 +104,8 @@ class ConfigLoader:
*CONFIG_BASE_OVERRIDES_AFTER_ENV,
]
for override in list_of_overrides:
- if override == runtime_manager.run_mode:
- match runtime_manager.run_mode:
- case RunMode.NORMAL:
- continue
- case RunMode.UNIT_TEST | RunMode.CI_TEST:
- override_path = os.path.join(os.getcwd(), "tests", f"pipelex_{override}.toml")
+ if override == runtime_manager.run_mode and runtime_manager.is_unit_testing:
+ override_path = os.path.join(os.getcwd(), "tests", f"pipelex_{override}.toml")
else:
override_path = os.path.join(os.getcwd(), "pipelex" if self.is_in_pipelex_config else "", f"pipelex_{override}.toml")
if override_dict := load_toml_from_path_if_exists(override_path):
diff --git a/pipelex/system/environment.py b/pipelex/system/environment.py
index dc85b2d3f..aebc2abef 100644
--- a/pipelex/system/environment.py
+++ b/pipelex/system/environment.py
@@ -42,3 +42,9 @@ def any_env_var_is_placeholder(keys: list[str]) -> bool:
def set_env(key: str, value: str) -> None:
os.environ[key] = value
+
+
+def is_env_var_truthy(key: str) -> bool:
+ """Return True if the env var is set and not a falsy sentinel ("false" or "0")."""
+ value = get_optional_env(key)
+ return (value is not None) and (value.lower() not in ["false", "0"])
diff --git a/pipelex/system/runtime.py b/pipelex/system/runtime.py
index b2b4a1465..b6ca033a6 100644
--- a/pipelex/system/runtime.py
+++ b/pipelex/system/runtime.py
@@ -1,9 +1,10 @@
from pydantic import BaseModel
-from pipelex.system.environment import get_optional_env
+from pipelex.system.environment import get_optional_env, is_env_var_truthy
from pipelex.types import StrEnum
RUN_MODE_ENV_VAR_KEY = "RUN_MODE"
+CODEX_CLOUD_ENV_VAR_KEY = "CODEX_CLOUD"
class IntegrationMode(StrEnum):
@@ -37,12 +38,17 @@ class RunMode(StrEnum):
NORMAL = "normal"
UNIT_TEST = "unit_test"
CI_TEST = "ci_test"
+ CODEX_CLOUD = "codex_cloud"
+ CODEX_CLOUD_TEST = "codex_cloud_test"
@classmethod
def get_from_env_var(cls) -> "RunMode":
if mode_str := get_optional_env(RUN_MODE_ENV_VAR_KEY):
return RunMode(mode_str)
- return RunMode.NORMAL
+ elif is_env_var_truthy(key=CODEX_CLOUD_ENV_VAR_KEY):
+ return RunMode.CODEX_CLOUD
+ else:
+ return RunMode.NORMAL
class WorkerMode(StrEnum):
@@ -109,24 +115,28 @@ class RuntimeManager(BaseModel):
match self.run_mode:
case RunMode.NORMAL:
return False
+ case RunMode.CODEX_CLOUD:
+ return False
case RunMode.UNIT_TEST:
return True
case RunMode.CI_TEST:
return True
+ case RunMode.CODEX_CLOUD_TEST:
+ return True
@property
def is_ci_testing(self) -> bool:
match self.run_mode:
case RunMode.NORMAL:
return False
+ case RunMode.CODEX_CLOUD:
+ return False
case RunMode.UNIT_TEST:
return False
case RunMode.CI_TEST:
return True
-
- @property
- def should_check_intermediate_configs(self) -> bool:
- return self.is_unit_testing
+ case RunMode.CODEX_CLOUD_TEST:
+ return True
runtime_manager = RuntimeManager()
diff --git a/pipelex/test_extras/shared_pytest_plugins.py b/pipelex/test_extras/shared_pytest_plugins.py
index 8adef1d3d..92d328ee2 100644
--- a/pipelex/test_extras/shared_pytest_plugins.py
+++ b/pipelex/test_extras/shared_pytest_plugins.py
@@ -5,8 +5,8 @@ from pytest import FixtureRequest, Parser
from pipelex.hub import get_console
from pipelex.pipe_run.pipe_run_params import PipeRunMode
-from pipelex.system.environment import is_env_var_set, set_env
-from pipelex.system.runtime import RunMode, runtime_manager
+from pipelex.system.environment import is_env_var_set, is_env_var_truthy, set_env
+from pipelex.system.runtime import CODEX_CLOUD_ENV_VAR_KEY, RunMode, runtime_manager
from pipelex.tools.misc.placeholder import make_placeholder_value, value_is_placeholder
# List of environment variables that may need placeholders in CI
@@ -41,6 +41,9 @@ ENV_VAR_KEYS_WHICH_MAY_NEED_PLACEHOLDERS_IN_CI = [
def set_run_mode():
if is_env_var_set(key="GITHUB_ACTIONS") or is_env_var_set(key="CI"):
runtime_manager.set_run_mode(run_mode=RunMode.CI_TEST)
+ elif is_env_var_truthy(key=CODEX_CLOUD_ENV_VAR_KEY):
+ # we're in codex cloud and this fixture is called by pytest, so we are testing in codex cloud
+ runtime_manager.set_run_mode(run_mode=RunMode.CODEX_CLOUD_TEST)
else:
runtime_manager.set_run_mode(run_mode=RunMode.UNIT_TEST)
diff --git a/pipelex/tools/log/log_config.py b/pipelex/tools/log/log_config.py
index 1595fb93a..572c020cd 100644
--- a/pipelex/tools/log/log_config.py
+++ b/pipelex/tools/log/log_config.py
@@ -1,5 +1,4 @@
import sys
-from enum import Enum
from typing import cast
from pydantic import Field, field_validator
@@ -27,7 +26,7 @@ class ProblemIds(StrEnum):
AZURE_OPENAI_NO_STREAM_OPTIONS = "Azure OpenAI no stream_options"
-class CallerInfoTemplate(Enum):
+class CallerInfoTemplate(StrEnum):
FILE_LINE = "file_line"
FILE_LINE_FUNC = "file_line_func"
FUNC = "func"
@@ -38,16 +37,21 @@ class CallerInfoTemplate(Enum):
@classmethod
def for_template_key(cls, key: "CallerInfoTemplate") -> str:
- templates: dict[CallerInfoTemplate, str] = {
- cls.FILE_LINE: "{file}:{line}",
- cls.FILE_LINE_FUNC: "{file}:{line} {func}",
- cls.FUNC: "{func}",
- cls.FILE_FUNC: "{file} {func}",
- cls.FUNC_LINE: "{func} {line}",
- cls.FUNC_MODULE: "{func} {module}",
- cls.FUNC_MODULE_LINE: "{func} {module} {line}",
- }
- return templates.get(key, "")
+ match key:
+ case cls.FILE_LINE:
+ return "{file}:{line}"
+ case cls.FILE_LINE_FUNC:
+ return "{file}:{line} {func}"
+ case cls.FUNC:
+ return "{func}"
+ case cls.FILE_FUNC:
+ return "{file} {func}"
+ case cls.FUNC_LINE:
+ return "{func} {line}"
+ case cls.FUNC_MODULE:
+ return "{func} {module}"
+ case cls.FUNC_MODULE_LINE:
+ return "{func} {module} {line}"
class RichLogConfig(ConfigModel):
diff --git a/pipelex/tools/misc/pretty.py b/pipelex/tools/misc/pretty.py
index ef6cf3383..33dfd279e 100644
--- a/pipelex/tools/misc/pretty.py
+++ b/pipelex/tools/misc/pretty.py
@@ -4,10 +4,10 @@ from io import StringIO
from typing import Any, ClassVar
from kajson import kajson
-from rich import print as rich_print
from rich.console import Console, Group
from rich.json import JSON
from rich.markdown import Markdown
+from rich.measure import Measurement
from rich.panel import Panel
from rich.pretty import Pretty
from rich.style import StyleType
@@ -73,8 +73,17 @@ def pretty_print(
inner_title: str | None = None,
border_style: StyleType | None = None,
width: int | None = None,
+ console_width: int | None = None,
):
- PrettyPrinter.pretty_print(content=content, title=title, subtitle=subtitle, inner_title=inner_title, border_style=border_style, width=width)
+ PrettyPrinter.pretty_print(
+ content=content,
+ title=title,
+ subtitle=subtitle,
+ inner_title=inner_title,
+ border_style=border_style,
+ width=width,
+ console_width=console_width,
+ )
def pretty_print_md(
@@ -84,10 +93,19 @@ def pretty_print_md(
inner_title: str | None = None,
border_style: StyleType | None = None,
width: int | None = None,
+ console_width: int | None = None,
):
width = width or PrettyPrinter.pretty_width()
md_content = Markdown(content)
- PrettyPrinter.pretty_print(content=md_content, title=title, subtitle=subtitle, inner_title=inner_title, border_style=border_style, width=width)
+ PrettyPrinter.pretty_print(
+ content=md_content,
+ title=title,
+ subtitle=subtitle,
+ inner_title=inner_title,
+ border_style=border_style,
+ width=width,
+ console_width=console_width,
+ )
class PrettyPrinter:
@@ -102,6 +120,7 @@ class PrettyPrinter:
inner_title: str | None = None,
border_style: StyleType | None = None,
width: int | None = None,
+ console_width: int | None = None,
):
match cls.mode:
case PrettyPrintMode.RICH:
@@ -112,9 +131,10 @@ class PrettyPrinter:
inner_title=inner_title,
border_style=border_style,
width=width,
+ console_width=console_width,
)
case PrettyPrintMode.POOR:
- cls.pretty_print_without_rich(content=content, title=title, subtitle=subtitle, inner_title=inner_title)
+ cls.pretty_print_without_rich(content=content, title=title, subtitle=subtitle, inner_title=inner_title, console_width=console_width)
@classmethod
def pretty_print_using_rich(
@@ -125,12 +145,19 @@ class PrettyPrinter:
inner_title: str | None = None,
border_style: StyleType | None = None,
width: int | None = None,
+ console_width: int | None = None,
):
panel = cls.make_pretty_panel(
- content=content, title=title, subtitle=subtitle, inner_title=inner_title, border_style=border_style, width=width
+ content=content,
+ title=title,
+ subtitle=subtitle,
+ inner_title=inner_title,
+ border_style=border_style,
+ width=width,
+ console_width=console_width,
)
- rich_print("", panel, "", sep="\n")
+ Console(width=console_width).print("", panel, "", sep="\n")
@classmethod
def pretty_width(cls, width: int | None = None, depth: int | None = None) -> int:
@@ -153,8 +180,19 @@ class PrettyPrinter:
inner_title: str | None = None,
border_style: StyleType | None = None,
width: int | None = None,
+ console_width: int | None = None,
) -> Panel:
pretty = cls.make_pretty(content, inner_title=inner_title, depth=0)
+ # When width is not specified, measure the content to determine optimal console width
+ if width is None:
+ # Create a console to measure the panel
+ measure_console = Console(width=console_width)
+ measurement = Measurement.get(measure_console, measure_console.options, pretty)
+ # Use the maximum width that fits the content, with some buffer for panel border rendering
+ width = measurement.maximum + 4
+ # print(f"width: {width}")
+ if console_width is not None:
+ width = min(width, console_width)
return cls.wrap_in_panel(pretty=pretty, title=title, subtitle=subtitle, border_style=border_style, width=width)
@classmethod
@@ -256,6 +294,8 @@ class PrettyPrinter:
title: TextType | None = None,
subtitle: TextType | None = None,
inner_title: str | None = None,
+ width: int | None = None,
+ console_width: int | None = None,
):
if isinstance(content, str) and content.startswith(("http://", "https://")):
cls.pretty_print_url_without_rich(content=content, title=title, subtitle=subtitle)
@@ -265,7 +305,7 @@ class PrettyPrinter:
title_str += f"\n{subtitle!s}"
if inner_title:
title_str += f"\n{inner_title}"
- terminal_width = shutil.get_terminal_size().columns
+ terminal_width = console_width or shutil.get_terminal_size().columns
content_str = f"{content}"
# Split title into lines if it contains newlines
@@ -274,6 +314,8 @@ class PrettyPrinter:
# Calculate max content width based on longest title line
max_title_len = max(len(line) for line in title_lines) if title_lines else 0
max_content_width = terminal_width - max_title_len - 8 # Accounting for frame and padding
+ if width:
+ max_content_width = min(max_content_width, width)
wrapped_lines: list[str] = []
for line in content_str.splitlines():
while len(line) > max_content_width:
diff --git a/pyproject.toml b/pyproject.toml
index fae0d4f4d..6ac2b70a2 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -205,9 +205,9 @@ typeCheckingMode = "strict"
minversion = "9.0"
addopts = [
"--import-mode=importlib",
- "-ra", # Show all test outcomes (including skips)
+ "-ra", # Show all test outcomes (including skips)
"-m",
- "not (inference or llm or img_gen or extract or needs_output or pipelex_api)",
+ "not (inference or llm or img_gen or extract or pipelex_api)",
]
asyncio_default_fixture_loop_scope = "session"
xfail_strict = true
@@ -220,7 +220,6 @@ filterwarnings = [
"ignore:Use.*Types instead",
]
markers = [
- "needs_output: tests that need output to be displayed",
"inference: slow and costly due to inference calls",
"llm: slow and costly due to llm inference calls",
"img_gen: slow and costly due to image generation inference calls",
@@ -394,7 +393,7 @@ reports = false
disable = ["all"]
enable = [
"W0101", # Unreachable code: Used when there is some code behind a "return" or "raise" statement, which will never be accessed.
- "C0103", # invalid-name (naming convention)
+ "C0103", # invalid-name (naming convention)
]
ignore = [".venv", "__pycache__", "build", "dist", ".git"]
@@ -402,13 +401,13 @@ ignore = [".venv", "__pycache__", "build", "dist", ".git"]
# Variables / attributes / arguments: snake_case, not length 1
variable-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
argument-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
-attr-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
+attr-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
# Functions / methods: snake_case, not length 1
function-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
-method-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
+method-rgx = "^(?!.$)[a-z_][a-z0-9_]*$"
# Classes: CapWords, not length 1
-class-rgx = "^(?!.$)[A-Z_][a-zA-Z0-9_]*$"
+class-rgx = "^(?!.$)[A-Z_][a-zA-Z0-9_]*$"
typevar-rgx = "^[A-Za-z_][A-Za-z0-9_]*$"
good-names = ["_"]
diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/test_pipe_parallel_1.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/test_pipe_parallel_1.py
index 556023a55..4994996bb 100644
--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/test_pipe_parallel_1.py
+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/test_pipe_parallel_1.py
@@ -10,7 +10,7 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory
from pipelex.core.stuffs.stuff_factory import StuffFactory
from pipelex.hub import get_pipe_router, get_required_pipe
from pipelex.pipe_run.pipe_job_factory import PipeJobFactory
-from pipelex.pipe_run.pipe_run_params import PipeRunMode
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
from pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactory
from pipelex.pipeline.job_metadata import JobMetadata
from tests.integration.pipelex.pipes.controller.pipe_parallel.pipe_parallel import ContentAnalysis, DocumentInput, LengthAnalysis
@@ -18,6 +18,7 @@ from tests.integration.pipelex.pipes.controller.pipe_parallel.pipe_parallel impo
@pytest.mark.llm
@pytest.mark.inference
@pytest.mark.asyncio(loop_scope="class")
class TestPipeParallelDocumentAnalysis:
@pytest.mark.parametrize(
diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py
index 39cd263cf..66671ed17 100644
--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py
+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py
@@ -19,22 +19,22 @@ from pipelex.pipeline.job_metadata import JobMetadata
@pytest.mark.dry_runnable
@pytest.mark.inference
@pytest.mark.asyncio
-async def test_simple_text_sequence(pipe_run_mode: PipeRunMode, load_test_library: Callable[[list[Path]], None]):
- load_test_library([Path("tests/integration/pipelex/pipes/controller/pipe_sequence/")])
- """Test simple text processing sequence without batching."""
- # Create test input
- raw_text_stuff = StuffFactory.make_stuff(
- name="raw_text",
- concept=ConceptFactory.make(
- concept_code="RawText",
- domain="simple_text_processing",
- description="simple_text_processing.RawText",
- structure_class_name="TextContent",
- ),
- content=TextContent(text="This is some messy text with bad spacing."),
- )
-
- if pipe_run_mode == PipeRunMode.DRY:
+class TestPipeSequenceSimple:
+ async def test_simple_text_sequence(self, pipe_run_mode: PipeRunMode, load_test_library: Callable[[list[Path]], None]):
+ load_test_library([Path("tests/integration/pipelex/pipes/controller/pipe_sequence/")])
+ """Test simple text processing sequence without batching."""
+ # Create test input
+ raw_text_stuff = StuffFactory.make_stuff(
+ name="raw_text",
+ concept=ConceptFactory.make(
+ concept_code="RawText",
+ domain="simple_text_processing",
+ description="simple_text_processing.RawText",
+ structure_class_name="TextContent",
+ ),
+ content=TextContent(text="This is some messy text with bad spacing."),
+ )
+
# Create working memory with the required input for dry run
working_memory = WorkingMemoryFactory.make_from_multiple_stuffs([raw_text_stuff])
diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_with_input_memory.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_with_input_memory.py
index a3552075f..390215b0f 100644
--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_with_input_memory.py
+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_with_input_memory.py
@@ -3,6 +3,7 @@
import pytest
from pipelex import pretty_print
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
from pipelex.pipeline.execute import execute_pipeline
from tests.e2e.pipelex.pipes.pipe_controller.pipe_sequence.test_tweet import OptimizedTweet
@@ -83,13 +84,14 @@ What $20 decision are you avoiding right now? 👇
"""
@pytest.mark.llm
@pytest.mark.inference
@pytest.mark.asyncio
class TestPipeSequenceWithInputMemory:
"""Test pipe sequence functionality with input memory."""
- async def test_optimize_tweet_sequence_with_input_memory(self):
+ async def test_optimize_tweet_sequence_with_input_memory(self, pipe_run_mode: PipeRunMode):
"""Test the optimize_tweet_sequence pipeline using inputs parameter."""
# Execute the pipeline using inputs
pipe_output = await execute_pipeline(
@@ -105,7 +107,9 @@ class TestPipeSequenceWithInputMemory:
"content": SAMPLE_WRITING_STYLE,
},
},
+ pipe_run_mode=pipe_run_mode,
)
+ pretty_print(pipe_output, title="Pipe output for optimize_tweet_sequence")
# Get the optimized tweet
optimized_tweet = pipe_output.main_stuff_as(content_type=OptimizedTweet)
@@ -114,15 +118,11 @@ class TestPipeSequenceWithInputMemory:
assert pipe_output is not None
assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
- assert optimized_tweet is not None
+ assert optimized_tweet
assert isinstance(optimized_tweet, OptimizedTweet)
- assert len(optimized_tweet.text) > 0
-
- # Log output and generate report
- pretty_print(pipe_output, title="Pipe output for optimize_tweet_sequence")
-
- # Verify the optimized tweet is different from the draft
- assert optimized_tweet.text != SAMPLE_DRAFT_TWEET
- # Verify it's not empty
- assert len(optimized_tweet.text.strip()) > 0
- assert "Maria Rodriguez" in optimized_tweet.text
+ assert len(optimized_tweet.lead_tweet.strip()) > 0
+ match pipe_run_mode:
+ case PipeRunMode.DRY:
+ pass
+ case PipeRunMode.LIVE:
+ assert "Maria Rodriguez" in optimized_tweet.lead_tweet
diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_tweet.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_tweet.py
index 51df37c10..50db267e1 100644
--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_tweet.py
+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_tweet.py
@@ -1,7 +1,10 @@
-from pipelex.core.stuffs.text_content import TextContent
+from pydantic import Field
+from pipelex.core.stuffs.structured_content import StructuredContent
-class OptimizedTweet(TextContent):
+
+class OptimizedTweet(StructuredContent):
"""A tweet optimized for Twitter/X engagement following best practices."""
- # TBD
+ lead_tweet: str
+ follow_up_tweets: list[str] | None = Field(default_factory=list[str])
diff --git a/tests/e2e/pipelex/pipes/pipe_operators/pipe_llm_vision.py b/tests/e2e/pipelex/pipes/pipe_operators/pipe_llm_vision.py
index 4a813afcb..eb28e0e9e 100644
--- a/tests/e2e/pipelex/pipes/pipe_operators/pipe_llm_vision.py
+++ b/tests/e2e/pipelex/pipes/pipe_operators/pipe_llm_vision.py
@@ -9,9 +9,10 @@ class BasicDescription(StructuredContent):
description: str = Field(description="Concise description of the image")
-class VisionAnalysis(StructuredContent):
+class VisionAnalysisE2E(StructuredContent):
"""Structured analysis of visual content from an image."""
main_subject: str = Field(description="The primary subject or focus of the image")
category: str = Field(description="The category of the image")
description: str = Field(description="Concise description of the image")
+ time_scale: str = Field(description="The time scale of gant chart: 'days', 'weeks', 'months', 'years' or something else")
diff --git a/tests/e2e/pipelex/pipes/pipe_operators/test_pipe_llm_vision.py b/tests/e2e/pipelex/pipes/pipe_operators/test_pipe_llm_vision.py
index 4abe18d9e..79c174a0d 100644
--- a/tests/e2e/pipelex/pipes/pipe_operators/test_pipe_llm_vision.py
+++ b/tests/e2e/pipelex/pipes/pipe_operators/test_pipe_llm_vision.py
@@ -4,24 +4,29 @@ from pathlib import Path
import pytest
-from pipelex import pretty_print
+from pipelex import pretty_print, pretty_print_md
from pipelex.core.stuffs.image_content import ImageContent
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
from pipelex.pipeline.execute import execute_pipeline
+from tests.e2e.pipelex.pipes.pipe_operators.pipe_llm_vision import VisionAnalysisE2E
from tests.integration.pipelex.cogt.test_data import LLMVisionTestCases
+from tests.integration.pipelex.test_data import PipeTestCases
@pytest.mark.llm
@pytest.mark.inference
@pytest.mark.asyncio
class TestPipeLLMVision:
- async def test_describe_image(self):
+ async def test_describe_image_single(self, pipe_run_mode: PipeRunMode):
# Execute the pipeline with an image
pipe_output = await execute_pipeline(
- pipe_code="describe_image",
+ pipe_code="describe_image_e2e",
library_path=str(Path("tests/e2e/pipelex/pipes/pipe_operators")),
inputs={
- "image": ImageContent(url=LLMVisionTestCases.PATH_IMG_GANTT_1),
+ "image": ImageContent(url=LLMVisionTestCases.URL_CLOUDFRONT_ALAN_TURING),
},
+ pipe_run_mode=pipe_run_mode,
)
# Get the result as text
@@ -43,38 +48,39 @@ class TestPipeLLMVision:
@pytest.mark.parametrize(
"pipe_code",
[
- "describe_image_number_1_only",
- "describe_image_number_2_only",
+ "describe_image_number_1_only_e2e",
+ "describe_image_number_2_only_e2e",
],
)
- async def test_describe_images_discriminate(self, pipe_code: str):
+ async def test_describe_images_multiple(self, pipe_run_mode: PipeRunMode, pipe_code: str):
"""Test the describe_image pipeline with multiple images to discriminate."""
# Execute the pipeline with an image
pipe_output = await execute_pipeline(
pipe_code=pipe_code,
library_path=str(Path("tests/e2e/pipelex/pipes/pipe_operators")),
inputs={
- "image_a": ImageContent(url=LLMVisionTestCases.PATH_IMG_GANTT_1),
- "image_b": ImageContent(url=LLMVisionTestCases.PATH_IMG_JPEG_3),
+ "image_a": ImageContent(url=LLMVisionTestCases.URL_CLOUDFRONT_ALAN_TURING),
+ "image_b": ImageContent(url=PipeTestCases.URL_IMG_FASHION_PHOTO_1),
},
+ pipe_run_mode=pipe_run_mode,
)
- result = pipe_output.main_stuff
- pretty_print(result, title=f"Image Description ({pipe_code})")
+ description = pipe_output.main_stuff_as_str
+ pretty_print_md(description, title=f"Image Description ({pipe_code})")
- async def test_structured_analysis_of_image_with_gantt_chart(self):
+ async def test_structured_analysis_of_image_with_gantt_chart(self, pipe_run_mode: PipeRunMode):
"""Test vision with a more complex image (Gantt chart)."""
# Execute the pipeline with a complex image
pipe_output = await execute_pipeline(
- pipe_code="vision_analysis",
+ pipe_code="vision_analysis_e2e",
inputs={
- "image": ImageContent(url=LLMVisionTestCases.PATH_IMG_GANTT_1),
+ "image": ImageContent(url=PipeTestCases.URL_IMG_GANTT_1),
},
+ pipe_run_mode=pipe_run_mode,
)
# Get the result as text
- # result = pipe_output.main_stuff_as(content_type=VisionAnalysis)
- result = pipe_output.main_stuff
+ result = pipe_output.main_stuff_as(content_type=VisionAnalysisE2E)
# Log output
pretty_print(result, title="Gantt Chart Description")
diff --git a/tests/integration/pipelex/cogt/test_data.py b/tests/integration/pipelex/cogt/test_data.py
index bd13b9b6d..50e2b9459 100644
--- a/tests/integration/pipelex/cogt/test_data.py
+++ b/tests/integration/pipelex/cogt/test_data.py
@@ -125,14 +125,14 @@ How tall do you think he was when he was 12? and at 15?
[
("name: John, age: 30", Person(name="John", age=30)),
# ("Betty Draper, 51", Person(name="Betty Draper", age=51)),
- ("Whiskers, the cat", Pet(species=PetSpecies.CAT, name="Whiskers")),
- ("Whiskers, the dog", Pet(species=PetSpecies.DOG, name="Whiskers")),
+ ("Whiskers, a very nice cat", Pet(species=PetSpecies.CAT, name="Whiskers")),
+ ("Whiskers, a cute little dog", Pet(species=PetSpecies.DOG, name="Whiskers")),
],
[
# ("name: Alice, age: 25", Person(name="Alice", age=25)),
("My sister's plumber, Bob Smith, is 42", Employee(name="Bob Smith", age=42, job="plumber")),
- ("Fluffy, the hamster", Pet(species=PetSpecies.HAMSTER, name="Fluffy")),
- ("Rex, the dog", Pet(species=PetSpecies.DOG, name="Rex")),
+ ("Fluffy is a funny hamster", Pet(species=PetSpecies.HAMSTER, name="Fluffy")),
+ ("Rex is a big black dog", Pet(species=PetSpecies.DOG, name="Rex")),
],
]
diff --git a/tests/integration/pipelex/cogt/test_extract.py b/tests/integration/pipelex/cogt/test_extract.py
index b277a69ca..3972f840c 100644
--- a/tests/integration/pipelex/cogt/test_extract.py
+++ b/tests/integration/pipelex/cogt/test_extract.py
@@ -14,6 +14,7 @@ from tests.cases import ImageTestCases, PDFTestCases
@pytest.mark.inference
@pytest.mark.asyncio(loop_scope="class")
@pytest.mark.usefixtures("routing_profile_override")
[email protected]("ignore:Accessing the 'model_fields' attribute on the instance is deprecated:DeprecationWarning")
class TestExtract:
@pytest.mark.parametrize("file_path", PDFTestCases.DOCUMENT_FILE_PATHS)
async def test_extract_pdf_path(self, extract_handle: str, file_path: str):
diff --git a/tests/integration/pipelex/cogt/test_llm_gen_object.py b/tests/integration/pipelex/cogt/test_llm_gen_object.py
index 768df8a78..c00c72fd8 100644
--- a/tests/integration/pipelex/cogt/test_llm_gen_object.py
+++ b/tests/integration/pipelex/cogt/test_llm_gen_object.py
@@ -63,7 +63,7 @@ class TestLLMGenObject:
pytest.skip(f"'{llm_worker.desc}' does not support object generation")
expected_class = expected_instance.__class__
output = await llm_worker.gen_object(llm_job=llm_job, schema=expected_class)
- pretty_print(output)
+ pretty_print(output, title=f"Output from {llm_preset_id}")
assert isinstance(output, expected_class)
assert output.model_dump(serialize_as_any=True) == expected_instance.model_dump(serialize_as_any=True)
diff --git a/tests/integration/pipelex/fixtures/llm_fixtures.py b/tests/integration/pipelex/fixtures/llm_fixtures.py
index 024229b17..f93ebde90 100644
--- a/tests/integration/pipelex/fixtures/llm_fixtures.py
+++ b/tests/integration/pipelex/fixtures/llm_fixtures.py
@@ -2,6 +2,7 @@
import pytest
+from pipelex import log
from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.hub import get_model_deck
from tests.integration.pipelex.fixtures.routing_fixtures import ALL_BACKENDS, check_backend_supports_model
@@ -18,6 +19,25 @@ def is_llm_handle_supported_by_enabled_backends(llm_handle: str) -> bool:
return any(check_backend_supports_model(backend, llm_handle) for backend in ALL_BACKENDS)
+def is_llm_preset_supported(llm_preset_id: str) -> bool:
+ """Check if an LLM preset is supported by at least one enabled backend."""
+ llm_setting = get_model_deck().get_llm_setting(llm_choice=llm_preset_id)
+ model_handle = llm_setting.model
+ model_deck = get_model_deck()
+ inference_model = model_deck.get_optional_inference_model(model_handle=model_handle)
+ if inference_model is None:
+ return False
+ log.debug(f"Inference model found!! {inference_model}")
+ return True
+
+
+def is_llm_preset_supported_by_enabled_backends(llm_preset_id: str) -> bool:
+ """Check if an LLM preset is supported by at least one enabled backend."""
+ llm_setting = get_model_deck().get_llm_setting(llm_choice=llm_preset_id)
+ # return is_llm_handle_supported_by_enabled_backends(llm_setting.model)
+ return any(check_backend_supports_model(backend, llm_setting.model) for backend in ALL_BACKENDS)
+
+
# ================================================================================================
# LLM Handles by Manufacturer
# each can be supported by multiple backends
@@ -159,18 +179,6 @@ ALL_LLM_HANDLES = [
]
- params=[
- # "llm_for_testing_gen_text",
- # "llm_for_testing_gen_object",
- "llm_for_creative_writing",
- ],
-)
-def llm_preset_id(request: pytest.FixtureRequest) -> str:
- assert isinstance(request.param, str)
- return request.param
-
-
@pytest.fixture(
params=[
LLMJobParams(
@@ -198,6 +206,23 @@ def llm_handle(request: pytest.FixtureRequest) -> str:
return llm_handle_param
+ params=[
+ # "llm_for_testing_gen_text",
+ # "llm_for_testing_gen_object",
+ "llm_for_creative_writing",
+ ],
+)
+def llm_preset_id(request: pytest.FixtureRequest) -> str:
+ assert isinstance(request.param, str)
+ llm_preset_id_param = request.param
+ if not is_llm_preset_supported(llm_preset_id=llm_preset_id_param):
+ pytest.skip(f"LLM preset '{llm_preset_id_param}' not supported by any enabled backend")
+ # if not is_llm_preset_supported_by_enabled_backends(llm_preset_id=llm_preset_id_param):
+ # pytest.skip(f"LLM preset '{llm_preset_id_param}' not supported by any enabled backend")
+ return llm_preset_id_param
+
+
@pytest.fixture(
params=[
# "gpt-4o",
diff --git a/tests/integration/pipelex/kit/conftest.py b/tests/integration/pipelex/kit/conftest.py
new file mode 100644
index 000000000..09ddd2219
--- /dev/null
+++ b/tests/integration/pipelex/kit/conftest.py
@@ -0,0 +1,7 @@
+import pytest
+
+
[email protected](params=["pipelex_language", "coding_standards"])
+def agent_set(request: pytest.FixtureRequest) -> str:
+ assert isinstance(request.param, str)
+ return request.param
diff --git a/tests/integration/pipelex/kit/test_cursor_export.py b/tests/integration/pipelex/kit/test_cursor_export.py
deleted file mode 100644
index 2a74c6243..000000000
--- a/tests/integration/pipelex/kit/test_cursor_export.py
+++ /dev/null
@@ -1,46 +0,0 @@
-from pathlib import Path
-
-from pipelex.kit.cursor_export import export_cursor_rules
-from pipelex.kit.index_loader import load_index
-
-
-class TestCursorExport:
- """Test Cursor rules export functionality."""
-
- def test_export_cursor_rules_dry_run(self, tmp_path: Path):
- """Test Cursor export in dry-run mode."""
- idx = load_index()
- repo_root = tmp_path
-
- # Dry run should not create files
- export_cursor_rules(repo_root, idx, dry_run=True)
- cursor_rules_dir = repo_root / ".cursor" / "rules"
- assert not cursor_rules_dir.exists() or len(list(cursor_rules_dir.iterdir())) == 0
-
- def test_export_cursor_rules_creates_mdc_files(self, tmp_path: Path):
- """Test that Cursor export creates .mdc files."""
- idx = load_index()
- repo_root = tmp_path
-
- export_cursor_rules(repo_root, idx, dry_run=False)
-
- # Verify output directory exists and contains .mdc files
- cursor_rules_dir = repo_root / ".cursor" / "rules"
- assert cursor_rules_dir.exists()
- mdc_files = list(cursor_rules_dir.glob("*.mdc"))
- assert len(mdc_files) > 0, "Expected .mdc files to be created"
-
- def test_export_cursor_rules_have_front_matter(self, tmp_path: Path):
- """Test that exported .mdc files have YAML front-matter."""
- idx = load_index()
- repo_root = tmp_path
-
- export_cursor_rules(repo_root, idx, dry_run=False)
-
- # Check first .mdc file for front-matter
- cursor_rules_dir = repo_root / ".cursor" / "rules"
- mdc_files = list(cursor_rules_dir.glob("*.mdc"))
- if mdc_files:
- content = mdc_files[0].read_text(encoding="utf-8")
- assert content.startswith("---\n"), "Expected YAML front-matter to start with ---"
- assert "---\n" in content[4:], "Expected YAML front-matter to end with ---"
diff --git a/tests/integration/pipelex/kit/test_cursor_rules_update.py b/tests/integration/pipelex/kit/test_cursor_rules_update.py
new file mode 100644
index 000000000..d5a08a82d
--- /dev/null
+++ b/tests/integration/pipelex/kit/test_cursor_rules_update.py
@@ -0,0 +1,65 @@
+from pathlib import Path
+
+import pytest
+
+from pipelex.kit.cursor_rules import update_cursor_rules
+from pipelex.kit.exceptions import KitError
+from pipelex.kit.index_loader import load_index
+
+
+class TestUpdateCursorRules:
+ def test_update_cursor_rules_dry_run(self, tmp_path: Path, agent_set: str):
+ kit_index = load_index()
+ repo_root = tmp_path
+
+ update_cursor_rules(repo_root, kit_index, agent_set=agent_set, dry_run=True)
+ cursor_rules_dir = repo_root / ".cursor" / "rules"
+ assert not cursor_rules_dir.exists() or len(list(cursor_rules_dir.iterdir())) == 0
+
+ def test_update_cursor_rules_creates_mdc_files(self, tmp_path: Path, agent_set: str):
+ kit_index = load_index()
+ repo_root = tmp_path
+
+ update_cursor_rules(repo_root, kit_index, agent_set=agent_set, dry_run=False)
+
+ cursor_rules_dir = repo_root / ".cursor" / "rules"
+ assert cursor_rules_dir.exists()
+ mdc_files = list(cursor_rules_dir.glob("*.mdc"))
+ assert len(mdc_files) > 0, "Expected .mdc files to be created"
+
+ def test_update_cursor_rules_have_front_matter(self, tmp_path: Path, agent_set: str):
+ kit_index = load_index()
+ repo_root = tmp_path
+
+ update_cursor_rules(repo_root, kit_index, agent_set=agent_set, dry_run=False)
+
+ # Check first .mdc file for front-matter
+ cursor_rules_dir = repo_root / ".cursor" / "rules"
+ mdc_files = list(cursor_rules_dir.glob("*.mdc"))
+ if mdc_files:
+ content = mdc_files[0].read_text(encoding="utf-8")
+ assert content.startswith("---\n"), "Expected YAML front-matter to start with ---"
+ assert "---\n" in content[4:], "Expected YAML front-matter to end with ---"
+
+ def test_update_cursor_rules_with_agent_set(self, tmp_path: Path, agent_set: str):
+ kit_index = load_index()
+ repo_root = tmp_path
+
+ update_cursor_rules(repo_root, kit_index, agent_set=agent_set, dry_run=False)
+
+ cursor_rules_dir = repo_root / ".cursor" / "rules"
+ exported_files: set[str] = set()
+ for mdc_file in cursor_rules_dir.glob("*.mdc"):
+ exported_files.add(mdc_file.name)
+
+ expected_files: set[str] = set()
+ for file_name in kit_index.agent_rules.sets[agent_set]:
+ expected_files.add(f"{file_name.removesuffix('.md')}.mdc")
+
+ assert exported_files == expected_files
+
+ def test_update_cursor_rules_invalid_agent_set(self, tmp_path: Path):
+ kit_index = load_index()
+
+ with pytest.raises(KitError, match=r"Agent set 'unknown' not found in index.toml"):
+ update_cursor_rules(tmp_path, kit_index, agent_set="unknown", dry_run=True)
diff --git a/tests/integration/pipelex/kit/test_idempotency.py b/tests/integration/pipelex/kit/test_idempotency.py
new file mode 100644
index 000000000..c2359779c
--- /dev/null
+++ b/tests/integration/pipelex/kit/test_idempotency.py
@@ -0,0 +1,114 @@
+"""Test idempotency of single-file agent rules generation."""
+
+from pathlib import Path
+
+from pipelex.kit.index_loader import load_index
+from pipelex.kit.single_file_agent_rules import update_single_file_agent_rules
+
+
+class TestIdempotency:
+ def test_update_single_file_agent_rules_is_idempotent_empty_file(self, tmp_path: Path, agent_set: str):
+ """Test that running update twice on an empty file produces identical results."""
+ idx = load_index()
+
+ repo_root = tmp_path / "repo"
+ repo_root.mkdir()
+ target_file = repo_root / "test_target.md"
+
+ # Start with empty file
+ target_file.write_text("", encoding="utf-8")
+
+ test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
+
+ # First update
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ first_result = target_file.read_text(encoding="utf-8")
+
+ # Second update (should be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ second_result = target_file.read_text(encoding="utf-8")
+
+ # Third update (should also be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ third_result = target_file.read_text(encoding="utf-8")
+
+ assert first_result == second_result, "Second run produced different output than first run"
+ assert second_result == third_result, "Third run produced different output than second run"
+
+ def test_update_single_file_agent_rules_is_idempotent_file_without_h1(self, tmp_path: Path, agent_set: str):
+ """Test that running update twice on a file without H1 produces identical results."""
+ idx = load_index()
+
+ repo_root = tmp_path / "repo"
+ repo_root.mkdir()
+ target_file = repo_root / "test_target.md"
+
+ # Start with file that has content but no H1
+ target_file.write_text("Some existing content without heading.\n", encoding="utf-8")
+
+ test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
+
+ # First update
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ first_result = target_file.read_text(encoding="utf-8")
+
+ # Second update (should be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ second_result = target_file.read_text(encoding="utf-8")
+
+ # Third update (should also be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ third_result = target_file.read_text(encoding="utf-8")
+
+ assert first_result == second_result, "Second run produced different output than first run"
+ assert second_result == third_result, "Third run produced different output than second run"
+
+ def test_update_single_file_agent_rules_is_idempotent_file_with_h1(self, tmp_path: Path, agent_set: str):
+ """Test that running update twice on a file with existing H1 produces identical results without duplicate headings."""
+ idx = load_index()
+
+ repo_root = tmp_path / "repo"
+ repo_root.mkdir()
+ target_file = repo_root / "test_target.md"
+
+ # Start with file that has an H1 heading already
+ target_file.write_text("# My Existing Heading\n\nSome existing content.\n", encoding="utf-8")
+
+ test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
+
+ # First update
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ first_result = target_file.read_text(encoding="utf-8")
+
+ # Verify no heading_1 was added inside markers (since file already has H1)
+ assert first_result.count("# Pipelex Coding Rules") == 0, "Should not add heading_1 when file already has H1"
+ assert first_result.count("# My Existing Heading") == 1, "Original H1 should be preserved"
+
+ # Second update (should be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ second_result = target_file.read_text(encoding="utf-8")
+
+ # Third update (should also be identical)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
+ third_result = target_file.read_text(encoding="utf-8")
+
+ assert first_result == second_result, "Second run produced different output than first run"
+ assert second_result == third_result, "Third run produced different output than second run"
diff --git a/tests/integration/pipelex/kit/test_merged_rules.py b/tests/integration/pipelex/kit/test_merged_rules.py
index fdad74a6b..0ab6c993b 100644
--- a/tests/integration/pipelex/kit/test_merged_rules.py
+++ b/tests/integration/pipelex/kit/test_merged_rules.py
@@ -3,7 +3,7 @@
import pytest
from pipelex.kit.index_loader import load_index
-from pipelex.kit.targets_update import build_merged_rules
+from pipelex.kit.single_file_agent_rules import build_merged_rules
class TestMergedRules:
diff --git a/tests/integration/pipelex/kit/test_targets_update.py b/tests/integration/pipelex/kit/test_update_single_file_agent_rules.py
similarity index 67%
rename from tests/integration/pipelex/kit/test_targets_update.py
rename to tests/integration/pipelex/kit/test_update_single_file_agent_rules.py
index 36a23917d..e312353db 100644
--- a/tests/integration/pipelex/kit/test_targets_update.py
+++ b/tests/integration/pipelex/kit/test_update_single_file_agent_rules.py
@@ -2,14 +2,11 @@ from pathlib import Path
from pipelex.kit.index_loader import load_index
from pipelex.kit.markers import find_span
-from pipelex.kit.targets_update import build_merged_rules, update_targets
+from pipelex.kit.single_file_agent_rules import update_single_file_agent_rules
-class TestTargetsUpdate:
- """Test target file updating functionality."""
-
- def test_update_targets_dry_run(self, tmp_path: Path):
- """Test updating targets in dry-run mode."""
+class TestUpdateSingleFileAgentRules:
+ def test_update_single_file_agent_rules_dry_run(self, tmp_path: Path, agent_set: str):
idx = load_index()
# Create a temporary repo root with a target file
@@ -18,20 +15,20 @@ class TestTargetsUpdate:
target_file = repo_root / "test_target.md"
target_file.write_text("# Test\n\nOriginal content\n", encoding="utf-8")
- merged_rules = build_merged_rules(idx)
-
# Create a test target
test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
original_content = target_file.read_text(encoding="utf-8")
# Dry run should not modify file
- update_targets(repo_root, merged_rules, test_targets, dry_run=True, diff=False, backup=None)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=True, diff=False, backup=None
+ )
assert target_file.read_text(encoding="utf-8") == original_content
- def test_update_targets_inserts_with_markers(self, tmp_path: Path):
- """Test that update_targets inserts content with markers."""
+ def test_update_single_file_agent_rules_inserts_with_markers(self, tmp_path: Path, agent_set: str):
+ """Test that agent rules update inserts content with markers."""
idx = load_index()
repo_root = tmp_path / "repo"
@@ -39,11 +36,11 @@ class TestTargetsUpdate:
target_file = repo_root / "test_target.md"
target_file.write_text("# Test\n\nOriginal content\n", encoding="utf-8")
- merged_rules = build_merged_rules(idx)
-
test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
- update_targets(repo_root, merged_rules, test_targets, dry_run=False, diff=False, backup=None)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
updated_content = target_file.read_text(encoding="utf-8")
target = test_targets["test"]
@@ -56,8 +53,8 @@ class TestTargetsUpdate:
span = find_span(updated_content, target.marker_begin, target.marker_end)
assert span is not None
- def test_update_targets_replaces_existing_markers(self, tmp_path: Path):
- """Test that update_targets replaces content between existing markers."""
+ def test_update_single_file_agent_rules_replaces_existing_markers(self, tmp_path: Path, agent_set: str):
+ """Test that agent rules update replaces content between existing markers."""
idx = load_index()
repo_root = tmp_path / "repo"
@@ -70,8 +67,6 @@ class TestTargetsUpdate:
initial_content = f"# Test\n\n{marker_begin}\nOld content\n{marker_end}\n"
target_file.write_text(initial_content, encoding="utf-8")
- merged_rules = build_merged_rules(idx)
-
test_targets = {
"test": idx.agent_rules.targets["agents"].model_copy(
update={
@@ -82,7 +77,9 @@ class TestTargetsUpdate:
)
}
- update_targets(repo_root, merged_rules, test_targets, dry_run=False, diff=False, backup=None)
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=None
+ )
updated_content = target_file.read_text(encoding="utf-8")
@@ -93,8 +90,8 @@ class TestTargetsUpdate:
# Verify old content is replaced
assert "Old content" not in updated_content
- def test_update_targets_creates_backup(self, tmp_path: Path):
- """Test that update_targets creates backup files when requested."""
+ def test_update_single_file_agent_rules_creates_backup(self, tmp_path: Path, agent_set: str):
+ """Test that agent rules update creates backup files when requested."""
idx = load_index()
repo_root = tmp_path / "repo"
@@ -103,11 +100,11 @@ class TestTargetsUpdate:
original_content = "# Test\n\nOriginal content\n"
target_file.write_text(original_content, encoding="utf-8")
- merged_rules = build_merged_rules(idx)
-
test_targets = {"test": idx.agent_rules.targets["agents"].model_copy(update={"path": "test_target.md"})}
- update_targets(repo_root, merged_rules, test_targets, dry_run=False, diff=False, backup=".bak")
+ update_single_file_agent_rules(
+ repo_root=repo_root, kit_index=idx, agent_set=agent_set, targets=test_targets, dry_run=False, diff=False, backup=".bak"
+ )
# Verify backup exists
backup_file = target_file.with_suffix(target_file.suffix + ".bak")
diff --git a/tests/unit/pipelex/tools/config/test_load_config.py b/tests/integration/pipelex/system/test_load_config.py
similarity index 94%
rename from tests/unit/pipelex/tools/config/test_load_config.py
rename to tests/integration/pipelex/system/test_load_config.py
index 7d342eb7a..703fc808c 100644
--- a/tests/unit/pipelex/tools/config/test_load_config.py
+++ b/tests/integration/pipelex/system/test_load_config.py
@@ -6,7 +6,7 @@ from pipelex.tools.misc.toml_utils import load_toml_from_path
class TestLoadConfig:
def test_load_kit_config(self):
- # The base condif must validate
+ # The base config must validate
config = config_manager.load_config()
PipelexConfig.model_validate(config)
diff --git a/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_extract_skill_presets.py b/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_extract_skill_presets.py
index b69dca574..c448301a8 100644
--- a/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_extract_skill_presets.py
+++ b/tests/unit/pipelex/builder/pipe/pipe_operator/pipe_extract/test_extract_skill_presets.py
@@ -5,7 +5,7 @@ from pipelex.cogt.exceptions import ModelChoiceNotFoundError
from pipelex.cogt.models.model_deck_check import check_extract_choice_with_deck
-class TestExtractSkill:
+class TestSkillPresetsExtract:
"""Test that all ExtractSkill enum values are valid ExtractModelChoice values in the model deck."""
def test_all_extract_skills_are_valid_extract_choices(self):
diff --git a/tests/unit/test_pipelex_check_initialization.py b/tests/unit/pipelex/system/test_pipelex_check_initialization.py
similarity index 100%
rename from tests/unit/test_pipelex_check_initialization.py
rename to tests/unit/pipelex/system/test_pipelex_check_initialization.py
diff --git a/tests/unit/pipelex/tools/misc/conftest.py b/tests/unit/pipelex/tools/misc/conftest.py
new file mode 100644
index 000000000..e691d0d68
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/conftest.py
@@ -0,0 +1,19 @@
+import pytest
+
+from tests.unit.pipelex.tools.misc.test_data import Address, ComplexUser, UserPreferences
+
+
+def complex_user() -> ComplexUser:
+ """Create a complex nested object for testing."""
+ return ComplexUser(
+ name="John Doe",
+ age=30,
+ addresses=[
+ Address(street="123 Main St", city="Springfield", country="USA", postal_code="12345"),
+ Address(street="456 Side St", city="Brooklyn", country="USA"),
+ ],
+ preferences=UserPreferences(theme="light", notifications=False, tags=["python", "coding"]),
+ metadata={"last_login": "2024-03-20", "login_count": 42, "is_active": True},
+ )
diff --git a/tests/unit/pipelex/tools/misc/test_data.py b/tests/unit/pipelex/tools/misc/test_data.py
new file mode 100644
index 000000000..7ca651d3b
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_data.py
@@ -0,0 +1,23 @@
+from pydantic import BaseModel, Field
+
+
+class Address(BaseModel):
+ street: str
+ city: str
+ country: str
+ postal_code: str | None = None
+
+
+class UserPreferences(BaseModel):
+ theme: str = "dark"
+ notifications: bool = True
+ tags: list[str] = Field(default_factory=list)
+
+
+class ComplexUser(BaseModel):
+ name: str
+ age: int
+ email: str | None
+ addresses: list[Address]
+ preferences: UserPreferences
+ metadata: dict[str, str | int | bool] = Field(default_factory=dict)
diff --git a/tests/unit/pipelex/tools/misc/test_pretty.py b/tests/unit/pipelex/tools/misc/test_pretty.py
index 63c5afcfb..025075c9e 100644
--- a/tests/unit/pipelex/tools/misc/test_pretty.py
+++ b/tests/unit/pipelex/tools/misc/test_pretty.py
@@ -2,32 +2,10 @@ import re
import textwrap
import pytest
-from pydantic import BaseModel, Field
from pytest import CaptureFixture
-from pipelex.tools.misc.pretty import PrettyPrinter, pretty_print
-
-
-class Address(BaseModel):
- street: str
- city: str
- country: str
- postal_code: str | None = None
-
-
-class UserPreferences(BaseModel):
- theme: str = "dark"
- notifications: bool = True
- tags: list[str] = Field(default_factory=list)
-
-
-class ComplexUser(BaseModel):
- name: str
- age: int
- email: str | None
- addresses: list[Address]
- preferences: UserPreferences
- metadata: dict[str, str | int | bool] = Field(default_factory=dict)
+from pipelex.tools.misc.pretty import pretty_print
+from tests.unit.pipelex.tools.misc.conftest import ComplexUser
def remove_ansi_escape_codes(text: str) -> str:
@@ -35,9 +13,16 @@ def remove_ansi_escape_codes(text: str) -> str:
return ansi_escape.sub("", text)
+def widths() -> tuple[int, int]:
+ """Return tuple of (width, console_width) for pretty print tests."""
+ return (180, 200)
+
+
class TestPrettyPrint:
- def test_pretty_print_with_brackets_optional_edge_case(self, capsys: CaptureFixture[str]):
- pretty_print(content="Optional[float]", title="title")
+ def test_pretty_print_with_brackets_optional_edge_case(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="Optional[float]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -53,8 +38,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_brackets_basic(self, capsys: CaptureFixture[str]):
- pretty_print(content="SomethingBeforeBracketsAnd[SomethingBetweenBrackets]", title="title")
+ def test_pretty_print_with_brackets_basic(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="SomethingBeforeBracketsAnd[SomethingBetweenBrackets]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -71,8 +57,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_nested_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="List[Optional[int]]", title="title")
+ def test_pretty_print_with_nested_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="List[Optional[int]]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -89,8 +76,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_multiple_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="Union[str, List[int], Dict[str, Any]]", title="title")
+ def test_pretty_print_with_multiple_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="Union[str, List[int], Dict[str, Any]]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -107,8 +95,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_special_chars_and_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="@decorator[*args, **kwargs]", title="title")
+ def test_pretty_print_with_special_chars_and_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="@decorator[*args, **kwargs]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -125,8 +114,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_empty_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="List[]", title="title")
+ def test_pretty_print_with_empty_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="List[]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -143,8 +133,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_unmatched_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="Unmatched[bracket", title="title")
+ def test_pretty_print_with_unmatched_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="Unmatched[bracket", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -161,8 +152,9 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- def test_pretty_print_with_unicode_and_brackets(self, capsys: CaptureFixture[str]):
- pretty_print(content="🐍Python[版本3.11]", title="title")
+ def test_pretty_print_with_unicode_and_brackets(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ pretty_print(content="🐍Python[版本3.11]", title="title", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
@@ -179,34 +171,17 @@ class TestPrettyPrint:
assert output == expected_output, f"Output did not match expected:\n{output}"
- # This test needs output to be displayed, so we disabled it by default in pyproject.toml
- # So to run it you must bypass the marker restrictions defined in the pyproject.toml pytest section
- # and make sure you turn on outputs with -s like this:
- # pytest -m "" -k test_pretty_print_pydantic_object -s
- @pytest.mark.gha_disabled
- @pytest.mark.codex_disabled
- @pytest.mark.needs_output
- def test_pretty_print_pydantic_object(self, capsys: CaptureFixture[str]):
- # Create a complex nested object
- user = ComplexUser(
- name="John Doe",
- age=30,
- addresses=[
- Address(street="123 Main St", city="Springfield", country="USA", postal_code="12345"),
- Address(street="456 Side St", city="Brooklyn", country="USA"),
- ],
- preferences=UserPreferences(theme="light", notifications=False, tags=["python", "coding"]),
- metadata={"last_login": "2024-03-20", "login_count": 42, "is_active": True},
- )
-
- pretty_print(content=user, title="Complex User")
+ def test_pretty_print_pydantic_object(self, complex_user: ComplexUser, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+
+ pretty_print(content=complex_user, title="Complex User", width=width, console_width=console_width)
captured = capsys.readouterr()
output = remove_ansi_escape_codes(captured.out)
- expected_output = textwrap.dedent("""\
+ expected_output = textwrap.dedent("""
╭─ Complex User ──────────────────────────────────────────────────────────────────────────────────╮
+ │ │
│ ComplexUser( │
│ name='John Doe', │
│ age=30, │
@@ -222,39 +197,9 @@ class TestPrettyPrint:
│ ), │
│ metadata={'last_login': '2024-03-20', 'login_count': 42, 'is_active': True} │
│ ) │
+ │ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────╯
- """)
-
- assert output == expected_output, f"Make sure you enable pytest '-s' option. Output did not match expected:\n{output}"
-
-class TestPrettyPrintInSandbox:
- def test_empty_content(self, capsys: CaptureFixture[str]):
- PrettyPrinter.pretty_print_without_rich(content="")
-
- captured = capsys.readouterr()
- output = remove_ansi_escape_codes(captured.err)
-
- expected_output = textwrap.dedent("""\
- ╭────╮
- │ │
- ╰────╯
- """)
-
- assert output == expected_output, f"Output did not match expected:\n{output}"
-
- def test_none_content(self, capsys: CaptureFixture[str]):
- PrettyPrinter.pretty_print_without_rich(content=None, title="title", subtitle="subtitle")
-
- captured = capsys.readouterr()
- output = remove_ansi_escape_codes(captured.err)
-
- expected_output = textwrap.dedent("""\
- ╭────────────╮
- │ title: │
- │ subtitle: │
- │ None │
- ╰────────────╯
""")
- assert output == expected_output, f"Output did not match expected:\n{output}"
+ assert output == expected_output, f"Make sure you enable pytest '-s' option. Output did not match expected:\n{output}"
diff --git a/tests/unit/pipelex/tools/misc/test_pretty_without_rich.py b/tests/unit/pipelex/tools/misc/test_pretty_without_rich.py
new file mode 100644
index 000000000..46e5cf1aa
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_pretty_without_rich.py
@@ -0,0 +1,72 @@
+import re
+import textwrap
+
+import pytest
+from pytest import CaptureFixture
+
+from pipelex.tools.misc.pretty import PrettyPrinter
+from tests.unit.pipelex.tools.misc.conftest import ComplexUser
+
+
+def remove_ansi_escape_codes(text: str) -> str:
+ ansi_escape = re.compile(r"\x1B\[[0-?]*[ -/]*[@-~]")
+ return ansi_escape.sub("", text)
+
+
+def widths() -> tuple[int, int]:
+ """Return tuple of (width, console_width) for pretty print tests."""
+ return (100, 150)
+
+
+class TestPrettyPrintWithoutRich:
+ def test_pretty_without_rich_empty_content(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ PrettyPrinter.pretty_print_without_rich(content="", width=width, console_width=console_width)
+
+ captured = capsys.readouterr()
+ output = remove_ansi_escape_codes(captured.err)
+
+ expected_output = textwrap.dedent("""\
+ ╭────╮
+ │ │
+ ╰────╯
+ """)
+
+ assert output == expected_output, f"Output did not match expected:\n{output}"
+
+ def test_pretty_without_rich_none_content(self, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ PrettyPrinter.pretty_print_without_rich(content=None, title="title", subtitle="subtitle", width=width, console_width=console_width)
+
+ captured = capsys.readouterr()
+ output = remove_ansi_escape_codes(captured.err)
+
+ expected_output = textwrap.dedent("""\
+ ╭────────────╮
+ │ title: │
+ │ subtitle: │
+ │ None │
+ ╰────────────╯
+ """)
+
+ assert output == expected_output, f"Output did not match expected:\n{output}"
+
+ def test_pretty_without_rich_pydantic_object(self, complex_user: ComplexUser, widths: tuple[int, int], capsys: CaptureFixture[str]):
+ width, console_width = widths
+ PrettyPrinter.pretty_print_without_rich(content=complex_user, title="Complex User", width=width, console_width=console_width)
+
+ captured = capsys.readouterr()
+ output = remove_ansi_escape_codes(captured.err)
+
+ expected_output = textwrap.dedent("""\
+ ╭────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+ │ Complex User: │
+ │ name='John Doe' age=30 email='[email protected]' addresses=[Address(street='123 Main St', city='Sprin │
+ │ gfield', country='USA', postal_code='12345'), Address(street='456 Side St', city='Brooklyn', country │
+ │ ='USA', postal_code=None)] preferences=UserPreferences(theme='light', notifications=False, tags=['py │
+ │ thon', 'coding']) metadata={'last_login': '2024-03-20', 'login_count': 42, 'is_active': True} │
+ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+ """)
+
+ assert output == expected_output, f"Output did not match expected:\n{output}"
diff --git a/tests/unit/pipelex/tools/test_log_config.py b/tests/unit/pipelex/tools/test_log_config.py
new file mode 100644
index 000000000..381ca9681
--- /dev/null
+++ b/tests/unit/pipelex/tools/test_log_config.py
@@ -0,0 +1,31 @@
+import pytest
+
+from pipelex.tools.log.log_config import CallerInfoTemplate, LogConfig
+from pipelex.tools.log.log_levels import LogLevel
+
+
+class TestLogConfigUtilities:
+ @pytest.mark.parametrize(
+ ("template_key", "expected_template"),
+ [
+ (CallerInfoTemplate.FILE_LINE, "{file}:{line}"),
+ (CallerInfoTemplate.FILE_LINE_FUNC, "{file}:{line} {func}"),
+ (CallerInfoTemplate.FUNC, "{func}"),
+ (CallerInfoTemplate.FILE_FUNC, "{file} {func}"),
+ (CallerInfoTemplate.FUNC_LINE, "{func} {line}"),
+ (CallerInfoTemplate.FUNC_MODULE, "{func} {module}"),
+ (CallerInfoTemplate.FUNC_MODULE_LINE, "{func} {module} {line}"),
+ ],
+ )
+ def test_caller_info_template_mapping(self, template_key: CallerInfoTemplate, expected_template: str) -> None:
+ assert CallerInfoTemplate.for_template_key(template_key) == expected_template
+
+ def test_validate_package_log_levels_converts_strings_to_enum(self) -> None:
+ package_levels = {"pipelex": "DEBUG", "pipelex.tools": "INFO"}
+
+ validated_levels = LogConfig.validate_package_log_levels(package_levels)
+
+ assert validated_levels == {
+ "pipelex": LogLevel.DEBUG,
+ "pipelex.tools": LogLevel.INFO,
+ }
diff --git a/tests/unit/pipelex/tools/test_log_levels.py b/tests/unit/pipelex/tools/test_log_levels.py
new file mode 100644
index 000000000..548093dad
--- /dev/null
+++ b/tests/unit/pipelex/tools/test_log_levels.py
@@ -0,0 +1,42 @@
+import logging
+
+import pytest
+
+from pipelex.tools.log.log_levels import (
+ LOGGING_LEVEL_DEV,
+ LOGGING_LEVEL_OFF,
+ LOGGING_LEVEL_VERBOSE,
+ LogLevel,
+)
+
+
+class TestLogLevels:
+ @pytest.mark.parametrize(
+ ("log_level", "expected_int_level"),
+ [
+ (LogLevel.VERBOSE, LOGGING_LEVEL_VERBOSE),
+ (LogLevel.DEBUG, logging.DEBUG),
+ (LogLevel.DEV, LOGGING_LEVEL_DEV),
+ (LogLevel.INFO, logging.INFO),
+ (LogLevel.WARNING, logging.WARNING),
+ (LogLevel.ERROR, logging.ERROR),
+ (LogLevel.CRITICAL, logging.CRITICAL),
+ (LogLevel.OFF, LOGGING_LEVEL_OFF),
+ ],
+ )
+ def test_int_logging_level_returns_correct_value(self, log_level: LogLevel, expected_int_level: int) -> None:
+ assert log_level.int_logging_level == expected_int_level
+
+ @pytest.mark.parametrize(
+ ("raw_level", "expected_enum"),
+ [
+ (LOGGING_LEVEL_VERBOSE, LogLevel.VERBOSE),
+ (LOGGING_LEVEL_DEV, LogLevel.DEV),
+ (LOGGING_LEVEL_OFF, LogLevel.OFF),
+ (logging.CRITICAL + 1, LogLevel.OFF),
+ (logging.DEBUG, LogLevel.DEBUG),
+ (logging.INFO, LogLevel.INFO),
+ ],
+ )
+ def test_from_int_converts_to_enum(self, raw_level: int, expected_enum: LogLevel) -> None:
+ assert LogLevel.from_int(raw_level) == expected_enum