Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Updated rules for all
Louis Choquel34559533a5f0
diff --git a/.cursor/rules/coding_standards.mdc b/.cursor/rules/coding_standards.mdcindex 6c6d06f8f..83aeec6aa 100644--- a/.cursor/rules/coding_standards.mdc+++ b/.cursor/rules/coding_standards.mdc@@ -10,10 +10,16 @@ This document outlines the core coding standards, best practices, and quality co## Type Hints1. **Always Use Type Hints**- - Every function parameter must be typed- - Every function return must be typed- - Use type hints for all variables where type is not obvious- - Use types with Uppercase first letter (Dict[], List[], etc.)++ - Every function parameter must be typed+ - Every function return must be typed+ - Use type hints for all variables where type is not obvious+ - Use dict, list, tupele types with lowercase first letter: dict[], list[], tuple[]+ - Use type hints for all fields+ - Use the `|` syntax for union types (e.g `str | int`) and `| None` for optionals+ - Use Field(default_factory=...) for mutable defaults and if it's a list of something else than str, use `empty_list_factory_of()` to make a factory: `number_list: list[int] = Field(default_factory=empty_list_factory_of(int), description="A list of numbers")`+ - Use `BaseModel` and respect Pydantic v2 standards, in particular use the modern `ConfigDict` when needed, e.g. `model_config = ConfigDict(extra="forbid", strict=True)`+ - Keep models focused and single-purpose2. **StrEnum**- Import from `pipelex.types`:@@ -21,19 +27,34 @@ This document outlines the core coding standards, best practices, and quality cofrom pipelex.types import StrEnum```-## BaseModel Standards--- Respect Pydantic v2 standards-- Keep models focused and single-purpose-- Use descriptive field names-- Use type hints for all fields-- Document complex validations-- Use Optional[] for nullable fields-- Use Field(default_factory=...) for mutable defaults+3. **Self type**+ - Import from `pipelex.types`:+ ```python+ from pipelex.types import Self+ ```## Factory Pattern-- Use Factory Pattern for object creation when dealing with multiple implementations+ - Use Factory Pattern for object creation when dealing with multiple implementations+ - Our factory methods are named `make_from_...` and such++## Error Handling++ - Always catch exceptions at the place where you can add useful context to it.+ - Use try/except blocks with specific exceptions+ - Convert third-party exceptions to our custom ones+ - Never catch Exception, only catch specific exceptions+ - Always add `from exc` to the exception++ ```python+ try:+ from fal_client import AsyncClient as FalAsyncClient+ except ImportError as exc:+ raise MissingDependencyError(+ "fal-client", "fal",+ "The fal-client SDK is required to use FAL models."+ ) from exc+ ```## Documentation@@ -61,21 +82,6 @@ This document outlines the core coding standards, best practices, and quality co"""```-## Error Handling--1. **Graceful Error Handling**- - Use try/except blocks with specific exceptions- - Convert third-party exceptions to custom ones- ```python- try:- from fal_client import AsyncClient as FalAsyncClient- except ImportError as exc:- raise MissingDependencyError(- "fal-client", "fal",- "The fal-client SDK is required to use FAL models."- ) from exc- ```-## Code Quality Checks### Linting and Type Checkingdiff --git a/.cursor/rules/docs.mdc b/.cursor/rules/docs.mdcindex 31461aa94..ac9c5191e 100644--- a/.cursor/rules/docs.mdc+++ b/.cursor/rules/docs.mdc@@ -6,3 +6,7 @@ alwaysApply: falseWrite docs and answer questions about writing docs.We use Material for MkDocs. All markdown in our docs must be compatible with Material for MkDocs and done using best practices to get the best results with Material for MkDocs.++## MkDocs Markdown Requirements++- Always add a blank line before any bullet lists or numbered lists in MkDocs markdown.diff --git a/.cursor/rules/llms.mdc b/.cursor/rules/llms.mdcindex 7f1d75f7b..54de9adcf 100644--- a/.cursor/rules/llms.mdc+++ b/.cursor/rules/llms.mdc@@ -21,10 +21,10 @@ An llm_handle can be either:```toml[aliases]-best-claude = "claude-4.1-opus"-best-gemini = "gemini-2.5-pro"-best-mistral = "mistral-large"+base-claude = "claude-4-sonnet"base-gpt = "gpt-5"+base-gemini = "gemini-2.5-flash"+base-mistral = "mistral-medium"```The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.@@ -38,7 +38,7 @@ Here is an example of using an llm_handle to specify which LLM to use in a PipeLtype = "PipeLLM"definition = "Write text about Hello World."output = "Text"-llm = { llm_handle = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }+llm = { llm_handle = "gpt-5", temperature = 0.9 }prompt_template = """Write a haiku about Hello World."""@@ -52,7 +52,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (tempExamples:```toml-llm_to_reason = { llm_handle = "o4-mini", temperature = 1, max_tokens = "auto" }+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }```@@ -78,4 +78,4 @@ The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.-You can override the predefined llm presets in `.pipelex/inference/deck/overrides.toml`.+You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.diff --git a/.cursor/rules/pipelex.mdc b/.cursor/rules/pipelex.mdcindex 3020740a7..52e8a88de 100644--- a/.cursor/rules/pipelex.mdc+++ b/.cursor/rules/pipelex.mdc@@ -66,11 +66,11 @@ definition = "....."```The pipes will all have at least this base structure.-- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if pipeSequence) or of the conditionnal pipes (if pipeCondition).+- `inputs`: Dictionnary of key behing the variable used in the prompts, and the value behing the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditionnal pipes (if PipeCondition).So If you have this error:`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' •-variable='['ocr_input']'``-That means that the pipe validate_expense is missing the input `ocr_input` because one of the subpipe is needing it.+variable='['invoice']'``+That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it.NEVER WRITE THE INPUTS BY BREAKING THE LINE LIKE THIS:<NEVER DO THIS>@@ -164,7 +164,8 @@ Look at the Pipes we have in order to adapt it. Pipes are organized in two categ2. **Operators** - For specific tasks:- `PipeLLM` - Generate Text and Objects (include Vision LLM)- - `PipeOcr` - OCR Pipe+ - `PipeOcr` - Extract text and images from an image or a PDF+ - `PipeCompose` - For composing text using Jinja2 templates: supports html, markdown, mermaid, etc.- `PipeImgGen` - Generate Images- `PipeFunc` - For running classic python scripts@@ -354,20 +355,20 @@ prompt_template = "Describe what you see in this image"## Purpose-Extract text and images from an image or a PDF+The PipeOcr operator is used to extract text and images from an image or a PDF## Basic Usage-### Simple Text Generation+### Simple Text Extraction```plx[pipe.extract_info]type = "PipeOcr"definition = "extract the information"-inputs = { ocr_input = "PDF" } # or { ocr_input = "Image" } if its an image. This is the only input+inputs = { document = "PDF" } # or { image = "Image" } if it's an image. This is the only input.output = "Page"```-The input ALWAYS HAS TO BE `ocr_input` and the value is either of concept `Image` or `Pdf`.+Only one input is allowed and it must either be an `Image` or a `PDF`.The output concept `Page` is a native concept, with the structure `PageContent`:It corresponds to 1 page. Therefore, the PipeOcr is outputing a `ListContent` of `Page`@@ -384,6 +385,264 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.+# PipeCompose Guide++## Purpose++The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.++## Basic Usage++### Simple Template Composition+```plx+[pipe.compose_report]+type = "PipeCompose"+definition = "Compose a report using template"+inputs = { data = "ReportData" }+output = "Text"+jinja2 = """+# Report Summary++Based on the analysis:+@data++Generated on: {{ current_date }}+"""+```++### Using Named Templates+```plx+[pipe.use_template]+type = "PipeCompose"+definition = "Use a predefined template"+inputs = { content = "Text" }+output = "Text"+jinja2_name = "standard_report_template"+```++### CRM Email Template+```plx+[pipe.compose_follow_up_email]+type = "PipeCompose"+definition = "Compose a personalized follow-up email for CRM"+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }+output = "Text"+template_category = "html"+prompting_style = { tag_style = "square_brackets", text_format = "html" }+jinja2 = """+Subject: Following up on our $deal.product_name discussion++Hi $customer.first_name,++I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.++Based on our discussion, I understand that your key requirements are: $deal.customer_requirements++I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:++**Next Steps:**+- Schedule a demo tailored to your specific needs+- Provide you with a customized quote based on your requirements+- Connect you with our implementation team++Would you be available for a 30-minute call this week? I have openings on:+{% for slot in available_slots %}+- {{ slot }}+{% endfor %}++Looking forward to moving this forward together!++Best regards,+$sales_rep.name+$sales_rep.title+$sales_rep.phone | $sales_rep.email+"""+```++## Key Parameters++- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)+- `prompting_style`: Styling options for template rendering+- `extra_context`: Additional context variables for template++## Template Variables++Use the same variable insertion rules as PipeLLM:+- `@variable` for block insertion (multi-line content)+- `$variable` for inline insertion (short text)++# PipeImgGen Guide++## Purpose++The PipeImgGen operator is used to generate images using AI image generation models.++## Basic Usage++### Simple Image Generation+```plx+[pipe.generate_image]+type = "PipeImgGen"+definition = "Generate an image from prompt"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+```++### Using Image Generation Settings+```plx+[pipe.generate_photo]+type = "PipeImgGen"+definition = "Generate a high-quality photo"+inputs = { prompt = "ImgGenPrompt" }+output = "Photo"+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }+aspect_ratio = "16:9"+nb_steps = 8+```++### Multiple Image Generation+```plx+[pipe.generate_variations]+type = "PipeImgGen"+definition = "Generate multiple image variations"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+nb_output = 3+seed = "auto"+```++### Advanced Configuration+```plx+[pipe.generate_custom]+type = "PipeImgGen"+definition = "Generate image with custom settings"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+img_gen = "img_gen_preset_name" # Use predefined preset+aspect_ratio = "1:1"+quality = "hd"+background = "transparent"+output_format = "png"+is_raw = false+safety_tolerance = 3+```++## Key Parameters++### Image Generation Settings+- `img_gen`: ImgGenChoice (preset name or inline settings)+- `img_gen_handle`: Direct model handle (legacy)+- `quality`: Image quality ("standard", "hd")+- `nb_steps`: Number of generation steps+- `guidance_scale`: How closely to follow the prompt++### Output Configuration+- `nb_output`: Number of images to generate+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)+- `output_format`: File format ("png", "jpeg", "webp")+- `background`: Background type ("default", "transparent")++### Generation Control+- `seed`: Random seed (integer or "auto")+- `is_raw`: Whether to apply post-processing+- `is_moderated`: Enable content moderation+- `safety_tolerance`: Content safety level (1-6)++## Input Requirements++PipeImgGen requires exactly one input that must be either:+- An `ImgGenPrompt` concept+- A concept that refines `ImgGenPrompt`++The input can be named anything but must contain the prompt text for image generation.++# PipeFunc Guide++## Purpose++The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.++## Basic Usage++### Simple Function Call+```plx+[pipe.process_data]+type = "PipeFunc"+definition = "Process data using custom function"+inputs = { input_data = "DataType" }+output = "ProcessedData"+function_name = "process_data_function"+```++### File Processing Example+```plx+[pipe.read_file]+type = "PipeFunc"+definition = "Read file content"+inputs = { file_path = "FilePath" }+output = "FileContent"+function_name = "read_file_content"+```++## Key Parameters++- `function_name`: Name of the Python function to call (must be registered in func_registry)++## Function Requirements++The Python function must:++1. **Be registered** in the `func_registry`+2. **Accept `working_memory`** as a parameter:+ ```python+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:+ # Function implementation+ pass+ ```++3. **Return appropriate types**:+ - `StuffContent`: Single content object+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)+ - `str`: Simple string (becomes TextContent)++## Function Registration++Functions must be registered in the function registry before use:++```python+from pipelex.tools.func_registry import func_registry++@func_registry.register("my_function_name")+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:+ # Access inputs from working memory+ input_data = working_memory.get_stuff("input_name")++ # Process data+ result = process_logic(input_data.content)++ # Return result+ return MyResultContent(data=result)+```++## Working Memory Access++Inside the function, access pipeline inputs through working memory:++```python+async def process_function(working_memory: WorkingMemory) -> TextContent:+ # Get input stuff by name+ input_stuff = working_memory.get_stuff("input_name")++ # Access the content+ input_content = input_stuff.content++ # Process and return+ processed_text = f"Processed: {input_content.text}"+ return TextContent(text=processed_text)+```+This rule explains how to write prompt templates in PipeLLM definitions.## Insert stuff inside a tagged block@@ -427,17 +686,47 @@ Be sure to make the proper choice of prefix for each insertion.**DO NOT write "$topic" alone in an isolated line.****DO write things like "Write an essay about $topic" included in an actual sentence.**-# Example to execute a pipeline+# Guide to write an example to execute a pipeline++## Example to execute a pipeline with text output```pythonimport asynciofrom pipelex import pretty_print-from pipelex.hub import get_pipeline_tracker, get_report_delegatefrom pipelex.pipelex import Pipelexfrom pipelex.pipeline.execute import execute_pipeline-from pipelex.libraries.pipelines.examples.extract_gantt.gantt import GanttChart++async def hello_world() -> str:+ """+ This function demonstrates the use of a super simple Pipelex pipeline to generate text.+ """+ # Run the pipe+ pipe_output = await execute_pipeline(+ pipe_code="hello_world",+ )++ return pipe_output.main_stuff_as_str+++# start Pipelex+Pipelex.make()+# run sample using asyncio+output_text = asyncio.run(hello_world())+pretty_print(output_text, title="Your first Pipelex output")+```++## Example to execute a pipeline with structured output++```python+import asyncio++from pipelex import pretty_print+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execute import execute_pipeline++from pipelex_libraries.pipelines.examples.extract_gantt.gantt import GanttChartSAMPLE_NAME = "extract_gantt"IMAGE_URL = "assets/gantt/gantt_tree_house.png"@@ -462,15 +751,14 @@ async def extract_gantt(image_url: str) -> GanttChart:Pipelex.make()# run sample using asyncio-gantt_chart = asyncio.run(extract_gantt(IMAGE_URL))--# Display cost report (tokens used and cost)-get_report_delegate().generate_report()-# output results+gantt_chart = asyncio.run(extract_gantt(image_url=IMAGE_URL))pretty_print(gantt_chart, title="Gantt Chart")-get_pipeline_tracker().output_flowchart()```+## Setting up the input memory++### Explanation of input memory+The input memory is a dictionary of key-value pairs, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = Dict[str, Any] | StuffContent | List[Any] | str@@ -496,7 +784,7 @@ So here are a few concrete examples of calls to execute_pipeline with various wapipe_output = await execute_pipeline(pipe_code="power_extractor_dpe",input_memory={- "ocr_input": PDFContent(url=pdf_url),+ "document": PDFContent(url=pdf_url),},)@@ -537,6 +825,168 @@ So here are a few concrete examples of calls to execute_pipeline with various wa)```+## Using the outputs of a pipeline++All pipe executions return a `PipeOutput` object.+It's a BaseModel which contains the resulting working memory at the end of the execution and the pipeline run id.+It also provides a bunch of accessor functions and properties to unwrap the main stuff, which is the last stuff added to the working memory:++```python++class PipeOutput(BaseModel):+ working_memory: WorkingMemory = Field(default_factory=WorkingMemory)+ pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)++ @property+ def main_stuff(self) -> Stuff:+ ...++ def main_stuff_as_list(self, item_type: type[StuffContentType]) -> ListContent[StuffContentType]:+ ...++ def main_stuff_as_items(self, item_type: type[StuffContentType]) -> list[StuffContentType]:+ ...++ def main_stuff_as(self, content_type: type[StuffContentType]) -> StuffContentType:+ ...++ @property+ def main_stuff_as_text(self) -> TextContent:+ ...++ @property+ def main_stuff_as_str(self) -> str:+ ...++ @property+ def main_stuff_as_image(self) -> ImageContent:+ ...++ @property+ def main_stuff_as_text_and_image(self) -> TextAndImagesContent:+ ...++ @property+ def main_stuff_as_number(self) -> NumberContent:+ ...++ @property+ def main_stuff_as_html(self) -> HtmlContent:+ ...++ @property+ def main_stuff_as_mermaid(self) -> MermaidContent:+ ...+```++As you can see, you can extarct any variable from the output working memory.++### Getting the main stuff as a specific type++Simple text as a string:++```python+result = pipe_output.main_stuff_as_str+```+Structured object (BaseModel):++```python+result = pipe_output.main_stuff_as(content_type=GanttChart)+```++If it's a list, you can get a `ListContent` of the specific type.++```python+result_list_content = pipe_output.main_stuff_as_list(item_type=GanttChart)+```++or if you want, you can get the actual items as a regular python list:++```python+result_list = pipe_output.main_stuff_as_items(item_type=GanttChart)+```++---++## Rules to choose LLM models used in PipeLLMs.++### LLM Configuration System++In order to use it in a pipe, an LLM is referenced by its llm_handle (alias) and possibly by an llm_preset.+LLM configurations are managed through the new inference backend system with files located in `.pipelex/inference/`:++- **Model Deck**: `.pipelex/inference/deck/base_deck.toml` and `.pipelex/inference/deck/overrides.toml`+- **Backends**: `.pipelex/inference/backends.toml` and `.pipelex/inference/backends/*.toml`+- **Routing**: `.pipelex/inference/routing_profiles.toml`++### LLM Handles++An llm_handle can be either:+1. **A direct model name** (like "gpt-4o-mini", "claude-3-sonnet") - automatically available for all models loaded by the inference backend system+2. **An alias** - user-defined shortcuts that map to model names, defined in the `[aliases]` section:++```toml+[aliases]+base-claude = "claude-4-sonnet"+base-gpt = "gpt-5"+base-gemini = "gemini-2.5-flash"+base-mistral = "mistral-medium"+```++The system first looks for direct model names, then checks aliases if no direct match is found. The system handles model routing through backends automatically.++### Using an LLM Handle in a PipeLLM++Here is an example of using an llm_handle to specify which LLM to use in a PipeLLM:++```plx+[pipe.hello_world]+type = "PipeLLM"+definition = "Write text about Hello World."+output = "Text"+llm = { llm_handle = "gpt-5", temperature = 0.9 }+prompt_template = """+Write a haiku about Hello World.+"""+```++As you can see, to use the LLM, you must also indicate the temperature (float between 0 and 1) and max_tokens (either an int or the string "auto").++### LLM Presets++Presets are meant to record the choice of an llm with its hyper parameters (temperature and max_tokens) if it's good for a particular task. LLM Presets are skill-oriented.++Examples:+```toml+llm_to_reason = { llm_handle = "base-claude", temperature = 1 }+llm_to_extract_invoice = { llm_handle = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" }+```++The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this:++```plx+[pipe.extract_invoice]+type = "PipeLLM"+definition = "Extract invoice information from an invoice text transcript"+inputs = { invoice_text = "InvoiceText" }+output = "Invoice"+llm = "llm_to_extract_invoice"+prompt_template = """+Extract invoice information from this invoice:++The category of this invoice is: $invoice_details.category.++@invoice_text+"""+```++The setting here `llm = "llm_to_extract_invoice"` works because "llm_to_extract_invoice" has been declared as an llm_preset in the deck.+You must not use an LLM preset in a PipeLLM that does not exist in the deck. If needed, you can add llm presets.++You can override the predefined llm presets by setting them in `.pipelex/inference/deck/overrides.toml`.++---+ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.diff --git a/.cursor/rules/pytest.mdc b/.cursor/rules/pytest.mdcindex ca31b9632..53db2a504 100644--- a/.cursor/rules/pytest.mdc+++ b/.cursor/rules/pytest.mdc@@ -3,6 +3,12 @@ description:globs: tests/**/*.pyalwaysApply: false---+# Writing unit tests++## Unit test generalities++NEVER USE unittest.mock or MagicMock. YOU MUST USE pytest-mock instead.+These rules apply when writing unit tests.- Always use pytest@@ -30,9 +36,9 @@ Apply the appropriate markers:Several markers may be applied. For instance, if the test uses an LLM, then it uses inference, so you must mark with both `inference`and `llm`.-## Tips+### Important rules-- Never use the unittest.mock. Use pytest-mock+- Never use the unittest.mock. Use pytest-mock.## Test Class Structure@@ -61,9 +67,9 @@ class TestFooBar:Sometimes it can be convenient to access the test's name in its body, for instance to include into a job_id. To achieve that, add the argument `request: FixtureRequest` into the signature and then you can get the test name using `cast(str, request.node.originalname), # type: ignore`.-# Pipe tests+## Writing integration test to test pipes-## Required imports for pipe tests+### Required imports for pipe tests```pythonimport pytest@@ -82,7 +88,7 @@ from pipelex.core.pipes.pipe_run_params import PipeRunParamsfrom pipelex.pipe_works.pipe_router_protocol import PipeRouterProtocol```-## Pipe test implementation steps+### Pipe test implementation steps1. Create Stuff from blueprint:@@ -112,14 +118,7 @@ pipe_output = await pipe_router.run_pipe()```-4. Log output and generate report:--```python-pretty_print(pipe_output, title=f"Pipe output")-get_report_delegate().generate_report()-```--5. Basic assertions:+4. Basic assertions:```pythonassert pipe_output is not None@@ -127,7 +126,7 @@ assert pipe_output.working_memory is not Noneassert pipe_output.main_stuff is not None```-## Test Data Organization+### Test Data Organization- If it's not already there, create a `test_data.py` file in the test directory- Define test cases using `StuffBlueprint`:diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.mdindex fcbc54ea0..b5dc7580c 100644--- a/.github/copilot-instructions.md+++ b/.github/copilot-instructions.md@@ -525,6 +525,258 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.+## PipeCompose operator++The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.++### Basic Usage++Simple Template Composition:+```plx+[pipe.compose_report]+type = "PipeCompose"+definition = "Compose a report using template"+inputs = { data = "ReportData" }+output = "Text"+jinja2 = """+# Report Summary++Based on the analysis:+$data++Generated on: {{ current_date }}+"""+```++Using Named Templates:+```plx+[pipe.use_template]+type = "PipeCompose"+definition = "Use a predefined template"+inputs = { content = "Text" }+output = "Text"+jinja2_name = "standard_report_template"+```++CRM Email Template:+```plx+[pipe.compose_follow_up_email]+type = "PipeCompose"+definition = "Compose a personalized follow-up email for CRM"+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }+output = "Text"+template_category = "html"+prompting_style = { tag_style = "square_brackets", text_format = "html" }+jinja2 = """+Subject: Following up on our $deal.product_name discussion++Hi $customer.first_name,++I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.++Based on our discussion, I understand that your key requirements are: $deal.customer_requirements++I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:++**Next Steps:**+- Schedule a demo tailored to your specific needs+- Provide you with a customized quote based on your requirements+- Connect you with our implementation team++Would you be available for a 30-minute call this week? I have openings on:+{% for slot in available_slots %}+- {{ slot }}+{% endfor %}++Looking forward to moving this forward together!++Best regards,+$sales_rep.name+$sales_rep.title+$sales_rep.phone | $sales_rep.email+"""+```++### Key Parameters++- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)+- `prompting_style`: Styling options for template rendering+- `extra_context`: Additional context variables for template++### Template Variables++Use the same variable insertion rules as PipeLLM:+- `@variable` for block insertion (multi-line content)+- `$variable` for inline insertion (short text)++## PipeImgGen operator++The PipeImgGen operator is used to generate images using AI image generation models.++### Basic Usage++Simple Image Generation:+```plx+[pipe.generate_image]+type = "PipeImgGen"+definition = "Generate an image from prompt"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+```++Using Image Generation Settings:+```plx+[pipe.generate_photo]+type = "PipeImgGen"+definition = "Generate a high-quality photo"+inputs = { prompt = "ImgGenPrompt" }+output = "Photo"+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }+aspect_ratio = "16:9"+nb_steps = 8+```++Multiple Image Generation:+```plx+[pipe.generate_variations]+type = "PipeImgGen"+definition = "Generate multiple image variations"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+nb_output = 3+seed = "auto"+```++Advanced Configuration:+```plx+[pipe.generate_custom]+type = "PipeImgGen"+definition = "Generate image with custom settings"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+img_gen = "img_gen_preset_name" # Use predefined preset+aspect_ratio = "1:1"+quality = "hd"+background = "transparent"+output_format = "png"+is_raw = false+safety_tolerance = 3+```++### Key Parameters++**Image Generation Settings:**+- `img_gen`: ImgGenChoice (preset name or inline settings)+- `img_gen_handle`: Direct model handle (legacy)+- `quality`: Image quality ("standard", "hd")+- `nb_steps`: Number of generation steps+- `guidance_scale`: How closely to follow the prompt++**Output Configuration:**+- `nb_output`: Number of images to generate+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)+- `output_format`: File format ("png", "jpeg", "webp")+- `background`: Background type ("default", "transparent")++**Generation Control:**+- `seed`: Random seed (integer or "auto")+- `is_raw`: Whether to apply post-processing+- `is_moderated`: Enable content moderation+- `safety_tolerance`: Content safety level (1-6)++### Input Requirements++PipeImgGen requires exactly one input that must be either:+- An `ImgGenPrompt` concept+- A concept that refines `ImgGenPrompt`++The input can be named anything but must contain the prompt text for image generation.++## PipeFunc operator++The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.++### Basic Usage++Simple Function Call:+```plx+[pipe.process_data]+type = "PipeFunc"+definition = "Process data using custom function"+inputs = { input_data = "DataType" }+output = "ProcessedData"+function_name = "process_data_function"+```++File Processing Example:+```plx+[pipe.read_file]+type = "PipeFunc"+definition = "Read file content"+inputs = { file_path = "FilePath" }+output = "FileContent"+function_name = "read_file_content"+```++### Key Parameters++- `function_name`: Name of the Python function to call (must be registered in func_registry)++### Function Requirements++The Python function must:++1. **Be registered** in the `func_registry`+2. **Accept `working_memory`** as a parameter:+ ```python+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:+ # Function implementation+ pass+ ```++3. **Return appropriate types**:+ - `StuffContent`: Single content object+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)+ - `str`: Simple string (becomes TextContent)++### Function Registration++Functions must be registered in the function registry before use:++```python+from pipelex.tools.func_registry import func_registry++@func_registry.register("my_function_name")+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:+ # Access inputs from working memory+ input_data = working_memory.get_stuff("input_name")++ # Process data+ result = process_logic(input_data.content)++ # Return result+ return MyResultContent(data=result)+```++### Working Memory Access++Inside the function, access pipeline inputs through working memory:++```python+async def process_function(working_memory: WorkingMemory) -> TextContent:+ # Get input stuff by name+ input_stuff = working_memory.get_stuff("input_name")++ # Access the content+ input_content = input_stuff.content++ # Process and return+ processed_text = f"Processed: {input_content.text}"+ return TextContent(text=processed_text)+```+---## Rules to choose LLM models used in PipeLLMs.diff --git a/AGENTS.md b/AGENTS.mdindex fcbc54ea0..b5dc7580c 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -525,6 +525,258 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.+## PipeCompose operator++The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.++### Basic Usage++Simple Template Composition:+```plx+[pipe.compose_report]+type = "PipeCompose"+definition = "Compose a report using template"+inputs = { data = "ReportData" }+output = "Text"+jinja2 = """+# Report Summary++Based on the analysis:+$data++Generated on: {{ current_date }}+"""+```++Using Named Templates:+```plx+[pipe.use_template]+type = "PipeCompose"+definition = "Use a predefined template"+inputs = { content = "Text" }+output = "Text"+jinja2_name = "standard_report_template"+```++CRM Email Template:+```plx+[pipe.compose_follow_up_email]+type = "PipeCompose"+definition = "Compose a personalized follow-up email for CRM"+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }+output = "Text"+template_category = "html"+prompting_style = { tag_style = "square_brackets", text_format = "html" }+jinja2 = """+Subject: Following up on our $deal.product_name discussion++Hi $customer.first_name,++I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.++Based on our discussion, I understand that your key requirements are: $deal.customer_requirements++I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:++**Next Steps:**+- Schedule a demo tailored to your specific needs+- Provide you with a customized quote based on your requirements+- Connect you with our implementation team++Would you be available for a 30-minute call this week? I have openings on:+{% for slot in available_slots %}+- {{ slot }}+{% endfor %}++Looking forward to moving this forward together!++Best regards,+$sales_rep.name+$sales_rep.title+$sales_rep.phone | $sales_rep.email+"""+```++### Key Parameters++- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)+- `prompting_style`: Styling options for template rendering+- `extra_context`: Additional context variables for template++### Template Variables++Use the same variable insertion rules as PipeLLM:+- `@variable` for block insertion (multi-line content)+- `$variable` for inline insertion (short text)++## PipeImgGen operator++The PipeImgGen operator is used to generate images using AI image generation models.++### Basic Usage++Simple Image Generation:+```plx+[pipe.generate_image]+type = "PipeImgGen"+definition = "Generate an image from prompt"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+```++Using Image Generation Settings:+```plx+[pipe.generate_photo]+type = "PipeImgGen"+definition = "Generate a high-quality photo"+inputs = { prompt = "ImgGenPrompt" }+output = "Photo"+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }+aspect_ratio = "16:9"+nb_steps = 8+```++Multiple Image Generation:+```plx+[pipe.generate_variations]+type = "PipeImgGen"+definition = "Generate multiple image variations"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+nb_output = 3+seed = "auto"+```++Advanced Configuration:+```plx+[pipe.generate_custom]+type = "PipeImgGen"+definition = "Generate image with custom settings"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+img_gen = "img_gen_preset_name" # Use predefined preset+aspect_ratio = "1:1"+quality = "hd"+background = "transparent"+output_format = "png"+is_raw = false+safety_tolerance = 3+```++### Key Parameters++**Image Generation Settings:**+- `img_gen`: ImgGenChoice (preset name or inline settings)+- `img_gen_handle`: Direct model handle (legacy)+- `quality`: Image quality ("standard", "hd")+- `nb_steps`: Number of generation steps+- `guidance_scale`: How closely to follow the prompt++**Output Configuration:**+- `nb_output`: Number of images to generate+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)+- `output_format`: File format ("png", "jpeg", "webp")+- `background`: Background type ("default", "transparent")++**Generation Control:**+- `seed`: Random seed (integer or "auto")+- `is_raw`: Whether to apply post-processing+- `is_moderated`: Enable content moderation+- `safety_tolerance`: Content safety level (1-6)++### Input Requirements++PipeImgGen requires exactly one input that must be either:+- An `ImgGenPrompt` concept+- A concept that refines `ImgGenPrompt`++The input can be named anything but must contain the prompt text for image generation.++## PipeFunc operator++The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.++### Basic Usage++Simple Function Call:+```plx+[pipe.process_data]+type = "PipeFunc"+definition = "Process data using custom function"+inputs = { input_data = "DataType" }+output = "ProcessedData"+function_name = "process_data_function"+```++File Processing Example:+```plx+[pipe.read_file]+type = "PipeFunc"+definition = "Read file content"+inputs = { file_path = "FilePath" }+output = "FileContent"+function_name = "read_file_content"+```++### Key Parameters++- `function_name`: Name of the Python function to call (must be registered in func_registry)++### Function Requirements++The Python function must:++1. **Be registered** in the `func_registry`+2. **Accept `working_memory`** as a parameter:+ ```python+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:+ # Function implementation+ pass+ ```++3. **Return appropriate types**:+ - `StuffContent`: Single content object+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)+ - `str`: Simple string (becomes TextContent)++### Function Registration++Functions must be registered in the function registry before use:++```python+from pipelex.tools.func_registry import func_registry++@func_registry.register("my_function_name")+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:+ # Access inputs from working memory+ input_data = working_memory.get_stuff("input_name")++ # Process data+ result = process_logic(input_data.content)++ # Return result+ return MyResultContent(data=result)+```++### Working Memory Access++Inside the function, access pipeline inputs through working memory:++```python+async def process_function(working_memory: WorkingMemory) -> TextContent:+ # Get input stuff by name+ input_stuff = working_memory.get_stuff("input_name")++ # Access the content+ input_content = input_stuff.content++ # Process and return+ processed_text = f"Processed: {input_content.text}"+ return TextContent(text=processed_text)+```+---## Rules to choose LLM models used in PipeLLMs.diff --git a/CLAUDE.md b/CLAUDE.mdindex fcbc54ea0..b5dc7580c 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -525,6 +525,258 @@ class PageContent(StructuredContent): # CONCEPT IS "Page"- `text_and_images` are the text, and the related images found in the input image or PDF.- `page_view` is the screenshot of the whole pdf page/image.+## PipeCompose operator++The PipeCompose operator is used to compose text using Jinja2 templates. It supports various output formats including HTML, Markdown, Mermaid diagrams, and more.++### Basic Usage++Simple Template Composition:+```plx+[pipe.compose_report]+type = "PipeCompose"+definition = "Compose a report using template"+inputs = { data = "ReportData" }+output = "Text"+jinja2 = """+# Report Summary++Based on the analysis:+$data++Generated on: {{ current_date }}+"""+```++Using Named Templates:+```plx+[pipe.use_template]+type = "PipeCompose"+definition = "Use a predefined template"+inputs = { content = "Text" }+output = "Text"+jinja2_name = "standard_report_template"+```++CRM Email Template:+```plx+[pipe.compose_follow_up_email]+type = "PipeCompose"+definition = "Compose a personalized follow-up email for CRM"+inputs = { customer = "Customer", deal = "Deal", sales_rep = "SalesRep" }+output = "Text"+template_category = "html"+prompting_style = { tag_style = "square_brackets", text_format = "html" }+jinja2 = """+Subject: Following up on our $deal.product_name discussion++Hi $customer.first_name,++I hope this email finds you well! I wanted to follow up on our conversation about $deal.product_name from $deal.last_contact_date.++Based on our discussion, I understand that your key requirements are: $deal.customer_requirements++I'm excited to let you know that we can definitely help you achieve your goals. Here's what I'd like to propose:++**Next Steps:**+- Schedule a demo tailored to your specific needs+- Provide you with a customized quote based on your requirements+- Connect you with our implementation team++Would you be available for a 30-minute call this week? I have openings on:+{% for slot in available_slots %}+- {{ slot }}+{% endfor %}++Looking forward to moving this forward together!++Best regards,+$sales_rep.name+$sales_rep.title+$sales_rep.phone | $sales_rep.email+"""+```++### Key Parameters++- `jinja2`: Inline Jinja2 template (mutually exclusive with jinja2_name)+- `jinja2_name`: Name of a predefined template (mutually exclusive with jinja2)+- `template_category`: Template type ("llm_prompt", "html", "markdown", "mermaid", etc.)+- `prompting_style`: Styling options for template rendering+- `extra_context`: Additional context variables for template++### Template Variables++Use the same variable insertion rules as PipeLLM:+- `@variable` for block insertion (multi-line content)+- `$variable` for inline insertion (short text)++## PipeImgGen operator++The PipeImgGen operator is used to generate images using AI image generation models.++### Basic Usage++Simple Image Generation:+```plx+[pipe.generate_image]+type = "PipeImgGen"+definition = "Generate an image from prompt"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+```++Using Image Generation Settings:+```plx+[pipe.generate_photo]+type = "PipeImgGen"+definition = "Generate a high-quality photo"+inputs = { prompt = "ImgGenPrompt" }+output = "Photo"+img_gen = { img_gen_handle = "dall-e-3", quality = "hd" }+aspect_ratio = "16:9"+nb_steps = 8+```++Multiple Image Generation:+```plx+[pipe.generate_variations]+type = "PipeImgGen"+definition = "Generate multiple image variations"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+nb_output = 3+seed = "auto"+```++Advanced Configuration:+```plx+[pipe.generate_custom]+type = "PipeImgGen"+definition = "Generate image with custom settings"+inputs = { prompt = "ImgGenPrompt" }+output = "Image"+img_gen = "img_gen_preset_name" # Use predefined preset+aspect_ratio = "1:1"+quality = "hd"+background = "transparent"+output_format = "png"+is_raw = false+safety_tolerance = 3+```++### Key Parameters++**Image Generation Settings:**+- `img_gen`: ImgGenChoice (preset name or inline settings)+- `img_gen_handle`: Direct model handle (legacy)+- `quality`: Image quality ("standard", "hd")+- `nb_steps`: Number of generation steps+- `guidance_scale`: How closely to follow the prompt++**Output Configuration:**+- `nb_output`: Number of images to generate+- `aspect_ratio`: Image dimensions ("1:1", "16:9", "9:16", etc.)+- `output_format`: File format ("png", "jpeg", "webp")+- `background`: Background type ("default", "transparent")++**Generation Control:**+- `seed`: Random seed (integer or "auto")+- `is_raw`: Whether to apply post-processing+- `is_moderated`: Enable content moderation+- `safety_tolerance`: Content safety level (1-6)++### Input Requirements++PipeImgGen requires exactly one input that must be either:+- An `ImgGenPrompt` concept+- A concept that refines `ImgGenPrompt`++The input can be named anything but must contain the prompt text for image generation.++## PipeFunc operator++The PipeFunc operator is used to run custom Python functions within a pipeline. This allows integration of classic Python scripts and custom logic.++### Basic Usage++Simple Function Call:+```plx+[pipe.process_data]+type = "PipeFunc"+definition = "Process data using custom function"+inputs = { input_data = "DataType" }+output = "ProcessedData"+function_name = "process_data_function"+```++File Processing Example:+```plx+[pipe.read_file]+type = "PipeFunc"+definition = "Read file content"+inputs = { file_path = "FilePath" }+output = "FileContent"+function_name = "read_file_content"+```++### Key Parameters++- `function_name`: Name of the Python function to call (must be registered in func_registry)++### Function Requirements++The Python function must:++1. **Be registered** in the `func_registry`+2. **Accept `working_memory`** as a parameter:+ ```python+ async def my_function(working_memory: WorkingMemory) -> StuffContent | list[StuffContent] | str:+ # Function implementation+ pass+ ```++3. **Return appropriate types**:+ - `StuffContent`: Single content object+ - `list[StuffContent]`: Multiple content objects (becomes ListContent)+ - `str`: Simple string (becomes TextContent)++### Function Registration++Functions must be registered in the function registry before use:++```python+from pipelex.tools.func_registry import func_registry++@func_registry.register("my_function_name")+async def my_custom_function(working_memory: WorkingMemory) -> StuffContent:+ # Access inputs from working memory+ input_data = working_memory.get_stuff("input_name")++ # Process data+ result = process_logic(input_data.content)++ # Return result+ return MyResultContent(data=result)+```++### Working Memory Access++Inside the function, access pipeline inputs through working memory:++```python+async def process_function(working_memory: WorkingMemory) -> TextContent:+ # Get input stuff by name+ input_stuff = working_memory.get_stuff("input_name")++ # Access the content+ input_content = input_stuff.content++ # Process and return+ processed_text = f"Processed: {input_content.text}"+ return TextContent(text=processed_text)+```+---## Rules to choose LLM models used in PipeLLMs.