Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Feature/polish (#367)
Louis Choquel5108ca510fb5
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 1c981f75e..43c1a705d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -245,7 +245,7 @@ This is all in the spirit of making Pipelex a declarative language, where you exWe've completely redesigned how LLMs are configured and accessed in Pipelex, making it more flexible and easier to get started:-- **Get started in seconds** with [Pipelex Inference](pages/configuration/config-technical/inference-backend-config.md#pipelex-inference): Use a single API key to access all major LLM providers (OpenAI, Anthropic, Google, Mistral, and more)+- **Get started in seconds** with [Pipelex Inference](pages/configuration/config-technical/inference-backend-config.md): Use a single API key to access all major LLM providers (OpenAI, Anthropic, Google, Mistral, and more)- **Flexible backend configuration**: Configure multiple inference backends (Azure OpenAI, AWS Bedrock, Vertex AI, etc.) through simple TOML files in `.pipelex/inference/`- **Smart model routing**: Automatically route models to the right backend using [routing profiles](pages/configuration/config-technical/inference-backend-config.md#routing-profiles) with pattern matching- **User-friendly aliases**: Define shortcuts like `best-claude` → `claude-4.1-opus` with optional fallback chainsdiff --git a/docs/index.md b/docs/index.mdindex fe81ccce6..2d8356b3a 100644--- a/docs/index.md+++ b/docs/index.md@@ -2,21 +2,127 @@title: "Open-source AI workflows"----+-# Easily build repeatable AI workflows+# Build reliable AI workflows in minutes-Pipelex is an open-source language specifically designed to build reliable, repeatable workflows. Stop wrestling with fragile workflows and start building AI systems you can trust.+## Install-## Ready to build something amazing?+```bash+pip install pipelex+pipelex init config+```-**Just want to see it in action?** Get up and running in minutes with our quick-start guide.+## Set your API key-[:material-rocket-launch: Install & Quick Start](pages/quick-start/index.md){ .md-button .md-button--primary .equal-width }+```bash+# Linux/MacOS+export PIPELEX_INFERENCE_API_KEY=###-**Want to understand the bigger picture?** Discover why Pipelex changes everything about AI development.+# Windows PowerShell+$env:PIPELEX_INFERENCE_API_KEY="###"-[:material-book-open: Read the Manifesto](manifesto.md){ .md-button .equal-width }-[:material-lightbulb: Explore the Paradigm](pages/pipelex-paradigm-for-repeatable-ai-workflows/index.md){ .md-button .equal-width }+# Windows CMD+set PIPELEX_INFERENCE_API_KEY=###+```++**Where to get an API key:** The `PIPELEX_INFERENCE_API_KEY` key provides access to all the AI models, you can get a free key from [our Discord](https://go.pipelex.com/discord). You can also use another AI routing service like [BlackBox AI](https://docs.blackbox.ai/), or bring your own API keys (OpenAI, Anthropic, Google, etc.), or run local AI (no key needed). See [Configure AI Providers](pages/setup/configure-ai-providers.md) for details. If you are using non-standard APIs, that's OK too, don't hesitate to join our [Discord](https://go.pipelex.com/discord) for guidance.++## Generate your first pipe++```bash+pipelex build pipe "Imagine a cute animal mascot for a startup based on its elevator pitch"+```++**Other useful use-cases for business:**++```bash+pipelex build pipe "Given an expense report, apply company rules"+pipelex build pipe "Take a CV in a PDF file, a Job offer text, and analyze if they match"+```++Each of these commands generates a complete production-ready script in our Pipelex language, saved as `.plx` file including domain definition, concepts, and the multiple _pipe_ steps to take to achieve the goal.++## Run your pipeline++**CLI:**++```bash+# Run a pipe by code+pipelex run <pipe_code>++# Run with inputs (JSON file containing input_memory dict)+pipelex run <pipe_code> --inputs input.json++# Run a bundle's main_pipe (auto-detected from .plx extension)+pipelex run bundle.plx --inputs input.json++# Customize output location+pipelex run <pipe_code> --output results/output.json+```++The `--inputs` file should be a JSON dictionary where keys are input variable names and values are the input data. For native types like Text, use strings directly. For structured types, provide objects matching the expected structure.++**Python:**++```python+import asyncio+from pipelex.pipeline.execute import execute_pipeline+from pipelex.pipelex import Pipelex++async def run_pipeline():+ pipe_output = await execute_pipeline(pipe_code="your_pipe_code")+ print(pipe_output.main_stuff_as_str)++Pipelex.make()+asyncio.run(run_pipeline())+```++## Easily iterate on your pipe++Now, thanks to our Pipelex language, and its high level of abstraction, you can directly edit the pipeline. It's pretty easy even for non-technical users. Better yet, you can get assisted in making changes with the help of your favorite AI coding assistant. To that end, we have prepared comprehensive guides for the most popular AI coding assistants and you can install them with one call:++```bash+pipelex kit rules+```++This installs Pipelex rules for Cursor, Claude Code, OpenAI Codex, GitHub Copilot, Windsurf, and Blackbox AI.++Now refine your pipeline with natural language:++- "Include confidence scores between 0 and 100 in the match analysis"+- "Write a recap email at the end"++---++## What is Pipelex?++Pipelex is an open-source Python framework for building **repeatable AI workflows**. Instead of cramming everything into one complex prompt, you break tasks into focused steps, each pipe handling one clear transformation.++Each pipe processes information using **Concepts** (typing with meaning) to ensure your pipelines make sense. The Pipelex language (`.plx` files) is simple and human-readable, even for non-technical users.++Each step can be structured and validated, so you benefit from the reliability of software, and the intelligence of AI.++---++## Next Steps++**Learn More:**++- [Full Tutorial](pages/quick-start/index.md) - Complete guide with examples+- [Cookbook Examples](pages/cookbook-examples/index.md) - Real-world patterns+- [Build Reliable AI Workflows](pages/build-reliable-ai-workflows-with-pipelex/kick-off-a-knowledge-pipeline-project.md) - Deep dive++**Understand the Philosophy:**++- [:material-book-open: Read the Manifesto](manifesto.md){ .md-button .equal-width }+- [:material-lightbulb: Explore the Paradigm](pages/pipelex-paradigm-for-repeatable-ai-workflows/index.md){ .md-button .equal-width }++**Configure:**++- [Configure AI Providers](pages/setup/configure-ai-providers.md) - API keys, local AI, model providers+- [Project Organization](pages/setup/project-organization.md) - Structure your Pipelex projects++[](https://github.com/Pipelex/pipelex-cookbook/)diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/llm-structured-generation-config.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/llm-structured-generation-config.mdindex 432ce9ab5..feaabbaaa 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/llm-structured-generation-config.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/llm-structured-generation-config.md@@ -72,10 +72,11 @@ The `is_default_text_then_structure` flag determines how LLM pipes generate struThe two-step process uses:1. The pipe's configured prompts for initial text generation-2. A specialized structure extraction prompt that can be customized:- - Through domain configuration (`domain.prompt_template_to_structure`)- - Through pipe configuration (`pipe.prompt_template_to_structure`)- - Falls back to base template if not specified+2. A specialized structure extraction prompt that can be customized in the `pipelex.toml` config in the `[cogt.llm_config.generic_templates]` section:++ - `structure_from_preliminary_text_system`: The system prompt for the second LLM call+ - `structure_from_preliminary_text_user`: The user prompt for the second LLM call+## Example Flowdiff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.mdindex ec8b5ee67..cf9a2866c 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md@@ -126,7 +126,6 @@ Analyze the document page shown in the image and explain how it relates to the p| `prompt_template` | string | A template for the user prompt. Use `$` for inline variables (e.g., `$topic`) and `@` to insert the content of an entire input (e.g., `@text_to_summarize`). **Note**: Do not use `@` or `$` for image variables. | No || `images` | list of strings | **Deprecated**: Use the `inputs` section to declare image inputs instead. | No || `structuring_method` | string | The method for generating structured output. Can be `direct` or `preliminary_text`. Defaults to the global configuration. | No |-| `prompt_template_to_structure` | string | The prompt template for the second step in `preliminary_text` mode. | No || `nb_output` | integer | Specifies exactly how many outputs to generate (e.g., `nb_output = 3` for exactly 3 outputs). Use when you need a fixed number of results. Mutually exclusive with `multiple_output`. | No || `multiple_output` | boolean | Controls output generation mode. Default is `false` (single output). Set to `true` for variable-length list generation when you need an indeterminate number of outputs. Mutually exclusive with `nb_output`. | No |diff --git a/docs/pages/configuration/config-technical/inference-backend-config.md b/docs/pages/configuration/config-technical/inference-backend-config.mdindex 26fcef9ea..7c091808c 100644--- a/docs/pages/configuration/config-technical/inference-backend-config.md+++ b/docs/pages/configuration/config-technical/inference-backend-config.md@@ -15,8 +15,6 @@ Get a single API key that works with all major providers (OpenAI, Anthropic, Goo- ✅ Automatic model routing- ✅ Free on Discord (limited time offer)-See [Pipelex Inference](#pipelex-inference) section below for setup details.-### Option B: Bring Your Own KeysUse your own API keys from individual providers for full control and direct billing. Ideal for production deployments with existing provider relationships.@@ -511,22 +509,3 @@ Common error types:- Keep base configurations unchanged to make upgrades easier- Version control your custom configurations- Share routing profiles and presets across your team--## Validation--Validate your inference configuration:--```bash-# Validate all configurations-pipelex validate all--# Check specific configuration-pipelex validate inference-```--The validation checks:-- Backend connectivity-- Model availability-- Routing consistency-- Preset validity-- Alias resolutiondiff --git a/docs/pages/quick-start/index.md b/docs/pages/quick-start/index.mdindex 7a6ea8258..06f19abf0 100644--- a/docs/pages/quick-start/index.md+++ b/docs/pages/quick-start/index.md@@ -1,128 +1,5 @@# Quick-start-Welcome to Pipelex! This guide will get you started building AI workflows in minutes.--## What is Pipelex?--Pipelex is an open-source Python framework for building **repeatable AI workflows**. Instead of cramming everything into one complex prompt, you break tasks into focused steps—each pipe handling one clear transformation. These workflows process information from **extraction** to **analysis** to **decision making**. The result? **Deterministic structure with adaptive intelligence**: the reliability of software with the flexibility of AI.--Each pipe processes information: it takes structured inputs and produces structured outputs. Pipelex uses **Concepts** (typing with meaning) to ensure your pipelines make sense. A "non-compete clause" and a "flower description" are both text, but they represent different concepts. Pipelex validates that pipes connect properly, catching errors before they happen.--The `.plx` language is simple and readable—even for non-technical users. You can generate pipelines with AI or write them yourself.--Learn more about the philosophy in [The Pipelex Paradigm](../pipelex-paradigm-for-repeatable-ai-workflows/index.md).-------## Installation--Install Pipelex with pip:--```bash-pip install pipelex-```--That's it! Now let's generate your first pipeline.-------## Generate Your First Pipeline with AI--The fastest way to get started is to generate pipelines using natural language. Pipelex can create complete, multi-step workflows for you.--### Step 1: Generate with Pipe Builder--The Pipe Builder transforms natural language descriptions into working `.plx` files. Here are some real-world examples:--**Example 1: Expense Report Validation**--```bash-pipelex build pipe "Given an expense report, apply company rules" -o expense_pipeline.plx-```--**Example 2: CV/Job Matching**--```bash-pipelex build pipe "Take a CV in a PDF file, a Job offer text, and analyze if they match" -o cv_matcher.plx-```--**Example 3: Compliance Analysis**--```bash-pipelex build pipe "Given an RFP PDF, build a compliance matrix" -o compliance_pipeline.plx-```--These commands generate complete `.plx` files containing:--- Domain definitions-- Concept declarations-- Multi-step pipe workflows--The Pipe Builder handles complexity automatically, creating pipelines with proper structure and validation.--!!! info "Pipe Builder is in Beta"- The Pipe Builder is currently in beta and improving rapidly. Expect frequent enhancements!--### Step 2: Install Agent Rules for Iteration--Once you have a generated pipeline, you can refine it using any AI coding assistant:--```bash-pipelex kit rules-```--This command installs Pipelex rules for:--- **Cursor**-- **Claude Code**-- **OpenAI Codex**-- **GitHub Copilot**-- **Windsurf**-- **Blackbox AI**--These rules teach your AI assistant Pipelex syntax, concepts, and best practices.--### Step 3: Iterate with Your AI Assistant--With the rules installed, use natural language to refine your pipeline:--- "Add a validation step to check for negative amounts"-- "Include confidence scores in the match analysis"-- "Add error handling for missing PDF files"-- "Create a summary report at the end"--Your AI assistant will understand Pipelex and make the changes correctly.--### Step 4: Test Your Pipeline--Run your pipeline from the command line or Python:--**CLI:**--```bash-pipelex run <pipe_code> --input-memory-from-json input.json-```--**Python:**--```python-import asyncio-from pipelex.pipeline.execute import execute_pipeline-from pipelex.pipelex import Pipelex--async def run_pipeline():- pipe_output = await execute_pipeline(pipe_code="your_pipe_code")- print(pipe_output.main_stuff_as_str)--Pipelex.make()-asyncio.run(run_pipeline())-```--!!! tip "Need API Access?"- To run these pipelines with LLMs, you'll need API access. We offer **free credits** for testing and developing, or you can bring your own keys, or run local AI. See the [Installation & Configuration](../installation/index.md#api-configuration) guide for all options.------## Understanding How It WorksReady to dive deeper? This section shows you how to manually create pipelines and understand the `.plx` language.@@ -144,8 +21,10 @@ domain = "characters"type = "PipeLLM"description = "Creates a character."output = "Text"-prompt = """You are a book writer. Your task is to create a character.-Think of it and then output the character description."""+prompt = """+You are a book writer. Your task is to create a character.+Think of it and then output the character description.+"""```This pipeline:@@ -198,8 +77,10 @@ type = "PipeLLM"description = "Create a character."output = "Text"model = { model = "gpt-4o-mini", temperature = 0.9, max_tokens = "auto" }-prompt = """You are a book writer. Your task is to create a character.-Think of it and then output the character description."""+prompt = """+You are a book writer. Your task is to create a character.+Think of it and then output the character description.+"""```#### Or Use an LLM Preset from the LLM Deck@@ -210,10 +91,12 @@ type = "PipeLLM"description = "Create a character."output = "Text"model = "llm_for_creative_writing"-prompt = """You are a book writer. Your task is to create a character.-Think of it and then output the character description."""+prompt = """+You are a book writer. Your task is to create a character.+Think of it and then output the character description.+"""-# The llm deck above is defined in `.pipelex/inference/deck/base_deck.toml` as:+# The llm choice above is defined in `.pipelex/inference/deck/base_deck.toml` as:# llm_for_creative_writing = { model = "best-claude", temperature = 0.9 }# it's a base preset that we provide. you can add your own presets, too.```@@ -279,8 +162,10 @@ Character = "A character in a fiction story" # <- Define your output concepttype = "PipeLLM"description = "Create a character. Get a structured result."output = "Character" # <- This is the output concept for your pipe-prompt = """You are a book writer. Your task is to create a character.-Think of it and then output the character description."""+prompt = """+You are a book writer. Your task is to create a character.+Think of it and then output the character description.+"""```💡 The concept name matches the class name (`Character`), so Pipelex automatically links them.diff --git a/docs/pages/installation/index.md b/docs/pages/setup/configure-ai-providers.mdsimilarity index 53%rename from docs/pages/installation/index.mdrename to docs/pages/setup/configure-ai-providers.mdindex 480262e9b..05d77b292 100644--- a/docs/pages/installation/index.md+++ b/docs/pages/setup/configure-ai-providers.md@@ -1,24 +1,6 @@-# Installation & Configuration+# Configure AI Providers-## Installation--Install Pipelex with pip:--```bash-pip install pipelex-```--Or use [uv](https://github.com/astral-sh/uv) for faster installs:--```bash-uv pip install pipelex-```--That's it! Pipelex is now installed. Continue to the [Quick-start Guide](../quick-start/index.md) to generate your first pipeline.-------## API Configuration+## Configure API AccessTo run pipelines with LLMs, you need to configure API access. **You have three options** - choose what works best for you:@@ -40,8 +22,8 @@ Get **free credits** for testing and development with a single API key that work- Request your key in the appropriate channel2. Create a `.env` file in your project root:- ```bash- echo "PIPELEX_INFERENCE_API_KEY=your-key-here" > .env+ ```env+ PIPELEX_INFERENCE_API_KEY=your-key-here```That's it! Your pipelines can now access any supported LLM.@@ -90,25 +72,28 @@ AWS_REGION=...You only need to add keys for the providers you plan to use.-**Configure the Inference Backend:**+**Enable Your Providers:**-When using your own keys, you need to tell Pipelex which backends to enable:+When using your own keys, enable the corresponding backends:-1. Initialize your configuration:+1. Initialize configuration:```bashpipelex init config```-2. Edit `.pipelex/inference/backends.toml` to enable your providers--For example, to use Google models:+2. Edit `.pipelex/inference/backends.toml`:```toml[google]enabled = true++[openai]+enabled = true++# Enable any providers you have keys for```-Learn more in our [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) guide.+See [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) for all options.### Option 3: Local AI (No API Keys Required)@@ -137,7 +122,7 @@ Configure these in `.pipelex/inference/backends.toml`. See our [Inference Backen----## Initialize Configuration+## Backend Configuration FilesTo set up Pipelex configuration files, run:@@ -149,64 +134,22 @@ This creates a `.pipelex/` directory with:- `pipelex.toml`: Feature flags, logging, cost reporting- `inference/`: LLM configuration and model presets+ - `backends.toml`: Enable/disable model providers+ - `deck/base_deck.toml`: LLM presets and aliases+ - `routing_profiles.toml`: Model routing configuration-Learn more in our [Configuration documentation](../configuration/index.md).-------## Project Organization--Pipelex automatically discovers `.plx` pipeline files anywhere in your project (excluding `.venv`, `.git`, `node_modules`, etc.).--**Recommended: Keep pipelines with related code**--```bash-your_project/-├── my_project/ # Your Python package-│ ├── finance/-│ │ ├── services.py-│ │ ├── invoices.plx # Pipeline with finance code-│ │ └── invoices_struct.py # Structure classes-│ └── legal/-│ ├── services.py-│ ├── contracts.plx # Pipeline with legal code-│ └── contracts_struct.py-├── .pipelex/ # Config at repo root-│ └── pipelex.toml-├── .env # API keys (git-ignored)-└── requirements.txt-```--**Alternative: Centralize pipelines**--```bash-your_project/-├── pipelines/-│ ├── invoices.plx-│ ├── contracts.plx-│ └── structures.py-└── .pipelex/- └── pipelex.toml-```--Learn more in our [Project Structure documentation](../build-reliable-ai-workflows-with-pipelex/kick-off-a-knowledge-pipeline-project.md).-------## Prerequisites--- **Python**: Version 3.10 or above-- **API Access**: One of the three options above (Pipelex Inference, your own keys, or local AI)+Learn more in our [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) guide.---## Next Steps-Now that you have Pipelex installed and configured:+Now that you have your backend configured:-1. **Start generating pipelines**: [Quick-start Guide](../quick-start/index.md)-2. **Explore examples**: [Cookbook Repository](https://github.com/Pipelex/pipelex-cookbook)-3. **Learn the concepts**: [The Pipelex Paradigm](../pipelex-paradigm-for-repeatable-ai-workflows/index.md)+1. **Organize your project**: [Project Organization](project-organization.md)+2. **Learn the concepts**: [Full Tutorial](../quick-start/index.md)+3. **Explore examples**: [Cookbook Repository](https://github.com/Pipelex/pipelex-cookbook)4. **Deep dive**: [Build Reliable AI Workflows](../build-reliable-ai-workflows-with-pipelex/kick-off-a-knowledge-pipeline-project.md)-💡 Need help? Check out our [Cookbook](https://github.com/Pipelex/pipelex-cookbook) for practical examples!+💡 For detailed backend configuration options, see [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md).+diff --git a/docs/pages/setup/project-organization.md b/docs/pages/setup/project-organization.mdnew file mode 100644index 000000000..2126a156e--- /dev/null+++ b/docs/pages/setup/project-organization.md@@ -0,0 +1,57 @@+# Project Organization++## Overview++Pipelex automatically discovers `.plx` pipeline files anywhere in your project (excluding `.venv`, `.git`, `node_modules`, etc.).++## Recommended: Keep pipelines with related code++```bash+your_project/+├── my_project/ # Your Python package+│ ├── finance/+│ │ ├── services.py+│ │ ├── invoices.plx # Pipeline with finance code+│ │ └── invoices_struct.py # Structure classes+│ └── legal/+│ ├── services.py+│ ├── contracts.plx # Pipeline with legal code+│ └── contracts_struct.py+├── .pipelex/ # Config at repo root+│ └── pipelex.toml+├── .env # API keys (git-ignored)+└── requirements.txt+```++## Alternative: Centralize pipelines++```bash+your_project/+├── pipelines/+│ ├── invoices.plx+│ ├── contracts.plx+│ └── structures.py+└── .pipelex/+ └── pipelex.toml+```++Learn more in our [Project Structure documentation](../build-reliable-ai-workflows-with-pipelex/kick-off-a-knowledge-pipeline-project.md).++---++## Prerequisites++- **Python**: Version 3.10 or above+- **API Access**: One of the three options from [Configure AI Providers](configure-ai-providers.md) (Pipelex Inference, your own keys, or local AI)++---++## Next Steps++Now that you understand project organization:++1. **Start building**: [Get Started](../../index.md)+2. **Learn the concepts**: [Full Tutorial](../quick-start/index.md)+3. **Explore examples**: [Cookbook Repository](https://github.com/Pipelex/pipelex-cookbook)+4. **Deep dive**: [Build Reliable AI Workflows](../build-reliable-ai-workflows-with-pipelex/kick-off-a-knowledge-pipeline-project.md)+diff --git a/generate_mascot_concept.json b/generate_mascot_concept.jsonnew file mode 100644index 000000000..a5758ecc0--- /dev/null+++ b/generate_mascot_concept.json@@ -0,0 +1,55 @@+{+ "root": {+ "elevator_pitch": {+ "stuff_code": "HkBMY",+ "stuff_name": "elevator_pitch",+ "concept": {+ "code": "Text",+ "domain": "native",+ "description": "A text",+ "structure_class_name": "TextContent",+ "refines": null+ },+ "content": {}+ },+ "mascot_characteristics": {+ "stuff_code": "gUStr",+ "stuff_name": "mascot_characteristics",+ "concept": {+ "code": "MascotCharacteristics",+ "domain": "startup_mascot_generation",+ "description": "Structured analysis of key attributes that should inform mascot design.",+ "structure_class_name": "MascotCharacteristics",+ "refines": null+ },+ "content": {}+ },+ "image_prompt": {+ "stuff_code": "BZoCP",+ "stuff_name": "image_prompt",+ "concept": {+ "code": "ImagePrompt",+ "domain": "startup_mascot_generation",+ "description": "A detailed textual description used to generate an image through AI image generation systems.",+ "structure_class_name": "TextContent",+ "refines": "native.Text"+ },+ "content": {}+ },+ "mascot_image": {+ "stuff_code": "Pe9HX",+ "stuff_name": "mascot_image",+ "concept": {+ "code": "MascotImage",+ "domain": "startup_mascot_generation",+ "description": "A visual representation of an animal character designed to represent a brand or organization.",+ "structure_class_name": "ImageContent",+ "refines": "native.Image"+ },+ "content": {}+ }+ },+ "aliases": {+ "main_stuff": "mascot_image"+ }+}\ No newline at end of filediff --git a/mkdocs.yml b/mkdocs.ymlindex a400b27e0..d4012bdde 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -86,7 +86,9 @@ nav:- Home:- Welcome: index.md- Quick-start: pages/quick-start/index.md- - Installation: pages/installation/index.md+ - Setup:+ - Configure AI Providers: pages/setup/configure-ai-providers.md+ - Project Organization: pages/setup/project-organization.md- Understand Pipelex:- The Knowledge Pipeline Manifesto: manifesto.md- The Pipelex Paradigm: pages/pipelex-paradigm-for-repeatable-ai-workflows/index.mddiff --git a/pipelex/builder/builder.plx b/pipelex/builder/builder.plxindex 5f425db42..44f44a44e 100644--- a/pipelex/builder/builder.plx+++ b/pipelex/builder/builder.plx@@ -9,7 +9,7 @@ PipelexBundleSpec = "A Pipelex bundle spec."ValidationResult = "Status (success or failure) and details of the validation failure if applicable."# PipeFailure = "Details of a single pipe failure during dry run."# DryRunResult = "A result of a dry run of a pipelex bundle spec."-DomainInformation = "A domain information object."+BundleHeaderSpec = "A domain information object."# ────────────────────────────────────────────────────────────────────────────────# Main@@ -26,24 +26,27 @@ steps = [{ pipe = "structure_concepts", result = "concept_specs" },{ pipe = "design_pipe_signatures", result = "pipe_signatures" },{ pipe = "detail_pipe_spec", batch_over = "pipe_signatures", batch_as = "pipe_signature", result = "pipe_specs" },- { pipe = "pipe_builder_domain_information", result = "domain_information" },+ { pipe = "write_bundle_header", result = "bundle_header_spec" },{ pipe = "assemble_pipelex_bundle_spec", result = "pipelex_bundle_spec" }]-[pipe.pipe_builder_domain_information]+[pipe.write_bundle_header]type = "PipeLLM"-description = "Turn the brief into a DomainInformation object."-inputs = { brief = "UserBrief" }-output = "DomainInformation"+description = "Write the bundle header."+inputs = { brief = "UserBrief", pipe_signatures = "PipeSignature" }+output = "BundleHeaderSpec"model = "llm_to_engineer"prompt = """Name and define the domain of this process:@brief+@pipe_signatures+For example, if the brief is about generating and analyzing a compliance matrix out of a RFP,-the domain would be "rfp_compliance" and the definition would be "Generating and analyzing compliance related to RFPs".+the domain would be "rfp_compliance" and the description would be "Generating and analyzing compliance related to RFPs".The domain name should be not more than 4 words, in snake_case.-For the definition, be concise.+For the description, be concise.+The main pipe is the one that will carry out the main task of the pipeline, it should be pretty obvious to identify."""[pipe.draft_the_plan]@@ -54,6 +57,7 @@ output = "PlanDraft"model = "llm_to_engineer"prompt = """Return a draft of a plan that narrates the pipeline as pseudo-steps (no code):+- Be clear which is the main pipe of the pipeline, don't write "main" in its pipe_code, but make it clear in its description.- Explicitly indicate when you are running things in sequence,or in parallel (several independant steps in parallel),or in batch (same operation applied to N elements of a list)@@ -187,6 +191,7 @@ You can use the native concepts for inputs/outputs as required: Text, Image, PDFDefine the contracts of the pipes to build:- For each pipe: give a unique snake_case pipe_code, a type and description, specify inputs (one or more) and output (one)- Add as much details as possible for the description.+- Be clear which is the main pipe of the pipeline, don't write "main" in its pipe_code, but make it clear in its description.Available pipe controllers:- PipeSequence: A pipe that executes a sequence of pipes: it needs to reference the pipes it will execute.@@ -216,6 +221,6 @@ Be smart about splitting the workflow into steps (sequence or parallel):[pipe.assemble_pipelex_bundle_spec]type = "PipeFunc"description = "Compile the pipelex bundle spec."-inputs = { pipe_specs = "PipeSpec", concept_specs = "ConceptSpec", domain_information = "DomainInformation" }+inputs = { pipe_specs = "PipeSpec", concept_specs = "ConceptSpec", bundle_header_spec = "BundleHeaderSpec" }output = "PipelexBundleSpec"function_name = "assemble_pipelex_bundle_spec"diff --git a/pipelex/builder/builder.py b/pipelex/builder/builder.pyindex bca9d8647..a6972df94 100644--- a/pipelex/builder/builder.py+++ b/pipelex/builder/builder.py@@ -1,47 +1,23 @@-from typing import TYPE_CHECKING, Annotated, cast+from pathlib import Path+from typing import TYPE_CHECKING, cast-from pydantic import ConfigDict, Field, ValidationError, field_validator+from pydantic import ValidationErrorfrom pipelex.builder.builder_errors import (- ConceptDefinitionErrorData,- ConceptFailure,- ConceptSpecError,- DomainFailure,PipeBuilderError,- PipeDefinitionErrorData,- PipeFailure,PipelexBundleError,PipelexBundleUnexpectedError,- PipeSpecError,- StaticValidationErrorData,- ValidateDryRunError,)+from pipelex.builder.builder_validation import document_pipe_failures_from_dry_run_blueprint, dry_run_bundle_blueprint+from pipelex.builder.bundle_header_spec import BundleHeaderSpec+from pipelex.builder.bundle_spec import PipelexBundleSpecfrom pipelex.builder.concept.concept_spec import ConceptSpec-from pipelex.builder.pipe.pipe_batch_spec import PipeBatchSpec-from pipelex.builder.pipe.pipe_compose_spec import PipeComposeSpec-from pipelex.builder.pipe.pipe_condition_spec import PipeConditionSpec-from pipelex.builder.pipe.pipe_extract_spec import PipeExtractSpec-from pipelex.builder.pipe.pipe_func_spec import PipeFuncSpec-from pipelex.builder.pipe.pipe_img_gen_spec import PipeImgGenSpec-from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec-from pipelex.builder.pipe.pipe_parallel_spec import PipeParallelSpec-from pipelex.builder.pipe.pipe_sequence_spec import PipeSequenceSpecfrom pipelex.builder.pipe.pipe_signature import PipeSpec-from pipelex.core.bundles.pipe_sorter import sort_pipes_by_dependencies-from pipelex.core.bundles.pipelex_bundle_blueprint import PipeBlueprintUnion, PipelexBundleBlueprint-from pipelex.core.concepts.concept_blueprint import ConceptBlueprint-from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.builder.pipe.pipe_spec_map import pipe_type_to_spec_class+from pipelex.builder.pipe.pipe_spec_union import PipeSpecUnion+from pipelex.core.interpreter import PipelexInterpreterfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.stuffs.list_content import ListContent-from pipelex.core.stuffs.structured_content import StructuredContent-from pipelex.exceptions import (- ConceptLoadingError,- DomainLoadingError,- PipeLoadingError,- StaticValidationError,-)-from pipelex.hub import get_library_manager-from pipelex.pipe_run.dry_run import dry_run_pipesfrom pipelex.system.registries.func_registry import pipe_funcfrom pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error@@ -49,138 +25,9 @@ if TYPE_CHECKING:from pipelex.core.stuffs.list_content import ListContent-class DomainInformation(StructuredContent):- domain: str = Field(description="Name of the domain of the knowledge work.")- description: str = Field(description="Definition of the domain of the knowledge work.")---PipeSpecUnion = Annotated[- PipeFuncSpec- | PipeImgGenSpec- | PipeComposeSpec- | PipeLLMSpec- | PipeExtractSpec- | PipeBatchSpec- | PipeConditionSpec- | PipeParallelSpec- | PipeSequenceSpec,- Field(discriminator="type"),-]---class PipelexBundleSpec(StructuredContent):- """Complete spec of a Pipelex bundle TOML definition.-- Represents the top-level structure of a Pipelex bundle, which defines a domain- with its concepts, pipes, and configuration. Bundles are the primary unit of- organization for Pipelex workflows, loaded from TOML files.-- Attributes:- domain: The domain identifier for this bundle in snake_case format.- Serves as the namespace for all concepts and pipes within.- description: Natural language description of the pipeline's purpose and functionality.- system_prompt: Default system prompt applied to all LLM pipes in the bundle- unless overridden at the pipe level.- system_prompt_to_structure: System prompt specifically for output structuring- operations across the bundle.- prompt_template_to_structure: Template for structuring prompts used in output- formatting operations.- concept: Dictionary of concept definitions used in this domain. Keys are concept- codes in PascalCase format, values are ConceptBlueprint instances or- string references to existing concepts.- pipe: Dictionary of pipe definitions for data transformation. Keys are pipe- codes in snake_case format, values are specific pipe spec types- (PipeLLM, PipeImgGen, PipeSequence, etc.).-- Validation Rules:- 1. Domain must be in valid snake_case format.- 2. Concept keys must be in PascalCase format.- 3. Pipe keys must be in snake_case format.- 4. Extra fields are forbidden (strict mode).- 5. Pipe types must match their blueprint discriminator.-- """-- model_config = ConfigDict(extra="forbid")-- domain: str- description: str | None = None- system_prompt: str | None = None-- concept: dict[str, ConceptSpec | str] | None = Field(default_factory=dict)-- pipe: dict[str, PipeSpecUnion] | None = Field(default_factory=dict)-- @field_validator("domain", mode="before")- @classmethod- def validate_domain_syntax(cls, domain: str) -> str:- DomainBlueprint.validate_domain_code(code=domain)- return domain-- def to_blueprint(self) -> PipelexBundleBlueprint:- concept: dict[str, ConceptBlueprint | str] | None = None-- if self.concept:- concept = {}- for concept_code, concept_spec_or_name in self.concept.items():- if isinstance(concept_spec_or_name, ConceptSpec):- try:- concept[concept_code] = concept_spec_or_name.to_blueprint()- except ValidationError as exc:- msg = f"Failed to create concept blueprint from spec for concept code {concept_code}: {format_pydantic_validation_error(exc)}"- concept_failure = ConceptFailure(concept_spec=concept_spec_or_name, error_message=msg)- raise ConceptSpecError(message=msg, concept_failure=concept_failure) from exc- else:- concept[concept_code] = ConceptBlueprint(description=concept_code, structure=concept_spec_or_name)-- pipe: dict[str, PipeBlueprintUnion] | None = None- if self.pipe:- # First, convert all specs to blueprints- pipe_blueprints: dict[str, PipeBlueprintUnion] = {}- for pipe_code, pipe_spec in self.pipe.items():- try:- pipe_blueprints[pipe_code] = pipe_spec.to_blueprint()- except ValidationError as exc:- msg = f"Failed to create pipe blueprint from spec for pipe code {pipe_code}: {format_pydantic_validation_error(exc)}"- pipe_failure = PipeFailure(pipe_spec=pipe_spec, error_message=msg)- raise PipeSpecError(message=msg, pipe_failure=pipe_failure) from exc-- # Then, sort blueprints by dependencies- try:- sorted_pipe_items = sort_pipes_by_dependencies(pipe_blueprints)- except Exception as exc:- msg = f"Failed to sort pipes by dependencies: {exc}"- raise PipeBuilderError(msg) from exc-- # Finally, create the ordered dict- pipe = dict(sorted_pipe_items)-- return PipelexBundleBlueprint(- domain=self.domain,- description=self.description,- prompt_template_to_structure=None,- system_prompt=self.system_prompt,- system_prompt_to_structure=None,- pipe=pipe,- concept=concept,- )--# # TODO: Put this in a factory. Investigate why it is necessary.def _convert_pipe_spec(pipe_spec: PipeSpecUnion) -> PipeSpecUnion:- pipe_type_to_class: dict[str, type] = {- "PipeFunc": PipeFuncSpec,- "PipeImgGen": PipeImgGenSpec,- "PipeCompose": PipeComposeSpec,- "PipeLLM": PipeLLMSpec,- "PipeExtract": PipeExtractSpec,- "PipeBatch": PipeBatchSpec,- "PipeCondition": PipeConditionSpec,- "PipeParallel": PipeParallelSpec,- "PipeSequence": PipeSequenceSpec,- }-- pipe_class = pipe_type_to_class.get(pipe_spec.type)+ pipe_class = pipe_type_to_spec_class.get(pipe_spec.type)if pipe_class is None:msg = f"Unknown pipe type: {pipe_spec.type}"raise PipeBuilderError(msg)@@ -209,7 +56,7 @@ async def assemble_pipelex_bundle_spec(working_memory: WorkingMemory) -> Pipelex)pipe_specs: list[PipeSpecUnion] = cast("ListContent[PipeSpecUnion]", working_memory.get_stuff(name="pipe_specs").content).items- domain_information = working_memory.get_stuff_as(name="domain_information", content_type=DomainInformation)+ bundle_header_spec = working_memory.get_stuff_as(name="bundle_header_spec", content_type=BundleHeaderSpec)# Properly validate and reconstruct concept specs to ensure proper Pydantic validationvalidated_concepts: dict[str, ConceptSpec | str] = {}@@ -224,112 +71,15 @@ async def assemble_pipelex_bundle_spec(working_memory: WorkingMemory) -> Pipelexraise PipeBuilderError(msg) from excreturn PipelexBundleSpec(- domain=domain_information.domain,- description=domain_information.description,+ domain=bundle_header_spec.domain,+ description=bundle_header_spec.description,+ system_prompt=bundle_header_spec.system_prompt,+ main_pipe=bundle_header_spec.main_pipe,concept=validated_concepts,pipe={pipe_spec.pipe_code: _convert_pipe_spec(pipe_spec) for pipe_spec in pipe_specs},)-async def validate_bundle_spec_from_memory(working_memory: WorkingMemory):- pipelex_bundle_spec = working_memory.get_stuff_as(name="pipelex_bundle_spec", content_type=PipelexBundleSpec)- await validate_bundle_spec(pipelex_bundle_spec=pipelex_bundle_spec)---async def validate_bundle_spec(pipelex_bundle_spec: PipelexBundleSpec):- library_manager = get_library_manager()- try:- pipelex_bundle_blueprint = pipelex_bundle_spec.to_blueprint()- except ConceptSpecError as concept_spec_error:- concept_failures = [concept_spec_error.concept_failure]- raise PipelexBundleError(message=concept_spec_error.message, concept_failures=concept_failures) from concept_spec_error- except PipeSpecError as pipe_spec_error:- pipe_failures = [pipe_spec_error.pipe_failure]- raise PipelexBundleError(message=pipe_spec_error.message, pipe_failures=pipe_failures) from pipe_spec_error-- try:- pipes = library_manager.load_from_blueprint(blueprint=pipelex_bundle_blueprint)- dry_run_result = await dry_run_pipes(pipes=pipes, raise_on_failure=True)- except StaticValidationError as static_validation_error:- static_validation_error_data = StaticValidationErrorData(- error_type=static_validation_error.error_type,- domain=static_validation_error.domain,- pipe_code=static_validation_error.pipe_code,- variable_names=static_validation_error.variable_names,- required_concept_codes=static_validation_error.required_concept_codes,- provided_concept_code=static_validation_error.provided_concept_code,- file_path=static_validation_error.file_path,- explanation=static_validation_error.explanation,- )- raise PipelexBundleError(- message=static_validation_error.desc(), static_validation_error=static_validation_error_data- ) from static_validation_error- except DomainLoadingError as domain_loading_error:- domain_failures = [DomainFailure(domain_code=domain_loading_error.domain_code, error_message=str(domain_loading_error))]- raise PipelexBundleError(message=domain_loading_error.message, domain_failures=domain_failures) from domain_loading_error- except ConceptLoadingError as concept_loading_error:- concept_def_error = concept_loading_error.concept_definition_error- concept_definition_error_data = ConceptDefinitionErrorData(- message=str(concept_def_error),- domain_code=concept_def_error.domain_code,- concept_code=concept_def_error.concept_code,- description=concept_def_error.description,- structure_class_python_code=concept_def_error.structure_class_python_code,- source=concept_def_error.source,- )- raise PipelexBundleError(- message=concept_loading_error.message, concept_definition_errors=[concept_definition_error_data]- ) from concept_loading_error- except PipeLoadingError as pipe_loading_error:- pipe_def_error = pipe_loading_error.pipe_definition_error- pipe_definition_error_data = PipeDefinitionErrorData(- message=str(pipe_def_error),- domain_code=pipe_def_error.domain_code,- pipe_code=pipe_def_error.pipe_code,- description=pipe_def_error.description,- source=pipe_def_error.source,- )- raise PipelexBundleError(message=pipe_loading_error.message, pipe_definition_errors=[pipe_definition_error_data]) from pipe_loading_error-- library_manager.remove_from_blueprint(blueprint=pipelex_bundle_blueprint)-- pipe_type_to_spec_class = {- "PipeFunc": PipeFuncSpec,- "PipeImgGen": PipeImgGenSpec,- "PipeCompose": PipeComposeSpec,- "PipeLLM": PipeLLMSpec,- "PipeExtract": PipeExtractSpec,- "PipeBatch": PipeBatchSpec,- "PipeCondition": PipeConditionSpec,- "PipeParallel": PipeParallelSpec,- "PipeSequence": PipeSequenceSpec,- }-- dry_run_pipe_failures: list[PipeFailure] = []- for pipe_code, dry_run_output in dry_run_result.items():- if dry_run_output.status.is_failure:- if not pipelex_bundle_spec.pipe:- msg = f"No pipes section found in bundle spec but we recorded a dry run failure for pipe '{pipe_code}'"- raise PipelexBundleUnexpectedError(message="No pipes section found in bundle spec")- if pipe_code not in pipelex_bundle_spec.pipe:- msg = f"Pipe '{pipe_code}' not found in bundle spec but we recorded a dry run failure for it"- raise PipelexBundleUnexpectedError(message=msg)-- pipe_spec = pipelex_bundle_spec.pipe[pipe_code]- spec_class = pipe_type_to_spec_class.get(pipe_spec.type)- if not spec_class:- msg = f"Unknown pipe type: {pipe_spec.type}"- raise ValidateDryRunError(msg)- pipe_spec = spec_class(**pipe_spec.model_dump(serialize_as_any=True))- pipe_failure = PipeFailure(- pipe_spec=pipe_spec,- error_message=dry_run_output.error_message or "",- )- dry_run_pipe_failures.append(pipe_failure)- if dry_run_pipe_failures:- raise PipelexBundleError(message="Pipes failed during dry run", pipe_failures=dry_run_pipe_failures)--async def reconstruct_bundle_with_pipe_fixes_from_memory(working_memory: WorkingMemory) -> PipelexBundleSpec:pipelex_bundle_spec = working_memory.get_stuff_as(name="pipelex_bundle_spec", content_type=PipelexBundleSpec)fixed_pipes_list = cast("ListContent[PipeSpecUnion]", working_memory.get_stuff(name="fixed_pipes").content)@@ -373,3 +123,38 @@ async def reconstruct_bundle_with_all_fixes(working_memory: WorkingMemory) -> Pipipelex_bundle_spec.concept[concept_code] = fixed_concept_blueprintreturn pipelex_bundle_spec+++async def load_pipe_from_bundle(bundle_path: str) -> str:+ """Load a bundle file and extract its main_pipe.++ Args:+ bundle_path: Path to the .plx bundle file.++ Returns:+ The pipe_code from the bundle's main_pipe.++ Raises:+ FileNotFoundError: If the bundle file does not exist.+ PipelexBundleError: If no main_pipe is declared or if pipes fail during dry run.+ PipeInputError: If there are input errors during dry run validation.+ """+ bundle_path_obj = Path(bundle_path)+ if not bundle_path_obj.exists():+ msg = f"Bundle file not found: {bundle_path}"+ raise FileNotFoundError(msg)++ interpreter = PipelexInterpreter(file_path=bundle_path_obj)+ bundle_blueprint = interpreter.make_pipelex_bundle_blueprint()++ if not bundle_blueprint.main_pipe:+ msg = f"Bundle '{bundle_path}' does not declare a main_pipe"+ raise PipelexBundleError(message=msg)++ dry_run_result = await dry_run_bundle_blueprint(bundle_blueprint=bundle_blueprint)+ pipe_failures = document_pipe_failures_from_dry_run_blueprint(bundle_blueprint=bundle_blueprint, dry_run_result=dry_run_result)+ if pipe_failures:+ msg = f"Pipes failed during dry run in bundle '{bundle_path}'"+ raise PipelexBundleError(message=msg, pipe_failures=pipe_failures)++ return bundle_blueprint.main_pipediff --git a/pipelex/builder/builder_loop.py b/pipelex/builder/builder_loop.pyindex c7d113f29..a703e492d 100644--- a/pipelex/builder/builder_loop.py+++ b/pipelex/builder/builder_loop.py@@ -3,13 +3,13 @@ from pipelex.builder.builder import (PipelexBundleSpec,PipeSpecUnion,reconstruct_bundle_with_pipe_fixes,- validate_bundle_spec,)from pipelex.builder.builder_errors import (PipelexBundleError,PipelexBundleNoFixForError,PipelexBundleUnexpectedError,)+from pipelex.builder.builder_validation import validate_bundle_specfrom pipelex.client.protocol import ImplicitMemoryfrom pipelex.core.pipes.pipe_blueprint import AllowedPipeCategoriesfrom pipelex.exceptions import StaticValidationErrorType@@ -34,7 +34,7 @@ class BuilderLoop:save_text_to_path(text=plx_content, path="results/generated_pipeline_1st_iteration.plx")try:- await validate_bundle_spec(pipelex_bundle_spec=pipelex_bundle_spec)+ await validate_bundle_spec(bundle_spec=pipelex_bundle_spec)except PipelexBundleError as bundle_error:pretty_print(bundle_error.as_structured_content(), title="Pipelex Bundle Error")pipelex_bundle_spec = self._fix_bundle_error(bundle_error=bundle_error, pipelex_bundle_spec=pipelex_bundle_spec)diff --git a/pipelex/builder/builder_validation.py b/pipelex/builder/builder_validation.pynew file mode 100644index 000000000..50f712491--- /dev/null+++ b/pipelex/builder/builder_validation.py@@ -0,0 +1,149 @@+from pipelex.builder.builder_errors import (+ ConceptDefinitionErrorData,+ ConceptSpecError,+ DomainFailure,+ PipeDefinitionErrorData,+ PipeFailure,+ PipelexBundleError,+ PipelexBundleUnexpectedError,+ PipeSpecError,+ StaticValidationErrorData,+ ValidateDryRunError,+)+from pipelex.builder.bundle_spec import PipelexBundleSpec+from pipelex.builder.pipe.pipe_spec_map import pipe_type_to_spec_class+from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.exceptions import (+ ConceptLoadingError,+ DomainLoadingError,+ PipeLoadingError,+ StaticValidationError,+)+from pipelex.hub import get_library_manager+from pipelex.pipe_run.dry_run import DryRunOutput, dry_run_pipes+++async def validate_bundle_spec_from_memory(working_memory: WorkingMemory):+ pipelex_bundle_spec = working_memory.get_stuff_as(name="pipelex_bundle_spec", content_type=PipelexBundleSpec)+ await validate_bundle_spec(bundle_spec=pipelex_bundle_spec)+++async def validate_bundle_spec(bundle_spec: PipelexBundleSpec):+ try:+ bundle_blueprint = bundle_spec.to_blueprint()+ except ConceptSpecError as concept_spec_error:+ concept_failures = [concept_spec_error.concept_failure]+ raise PipelexBundleError(message=concept_spec_error.message, concept_failures=concept_failures) from concept_spec_error+ except PipeSpecError as pipe_spec_error:+ pipe_failures = [pipe_spec_error.pipe_failure]+ raise PipelexBundleError(message=pipe_spec_error.message, pipe_failures=pipe_failures) from pipe_spec_error++ library_manager = get_library_manager()+ dry_run_result = await dry_run_bundle_blueprint(bundle_blueprint=bundle_blueprint)+ library_manager.remove_from_blueprint(blueprint=bundle_blueprint)++ dry_run_pipe_failures = extract_pipe_failures_from_dry_run_result(bundle_spec=bundle_spec, dry_run_result=dry_run_result)+ if dry_run_pipe_failures:+ raise PipelexBundleError(message="Pipes failed during dry run", pipe_failures=dry_run_pipe_failures)+++def extract_pipe_failures_from_dry_run_result(bundle_spec: PipelexBundleSpec, dry_run_result: dict[str, DryRunOutput]) -> list[PipeFailure]:+ dry_run_pipe_failures: list[PipeFailure] = []+ for pipe_code, dry_run_output in dry_run_result.items():+ if dry_run_output.status.is_failure:+ if not bundle_spec.pipe:+ msg = f"No pipes section found in bundle spec but we recorded a dry run failure for pipe '{pipe_code}'"+ raise PipelexBundleUnexpectedError(message="No pipes section found in bundle spec")+ if pipe_code not in bundle_spec.pipe:+ msg = f"Pipe '{pipe_code}' not found in bundle spec but we recorded a dry run failure for it"+ raise PipelexBundleUnexpectedError(message=msg)++ pipe_spec = bundle_spec.pipe[pipe_code]+ spec_class = pipe_type_to_spec_class.get(pipe_spec.type)+ if not spec_class:+ msg = f"Unknown pipe type: {pipe_spec.type}"+ raise ValidateDryRunError(msg)+ pipe_spec = spec_class(**pipe_spec.model_dump(serialize_as_any=True))+ pipe_failure = PipeFailure(+ pipe_spec=pipe_spec,+ error_message=dry_run_output.error_message or "",+ )+ dry_run_pipe_failures.append(pipe_failure)+ return dry_run_pipe_failures+++def document_pipe_failures_from_dry_run_blueprint(+ bundle_blueprint: PipelexBundleBlueprint, dry_run_result: dict[str, DryRunOutput]+) -> list[PipeFailure]:+ dry_run_pipe_failures: list[PipeFailure] = []+ for pipe_code, dry_run_output in dry_run_result.items():+ if dry_run_output.status.is_failure:+ if not bundle_blueprint.pipe:+ msg = f"No pipes section found in bundle spec but we recorded a dry run failure for pipe '{pipe_code}'"+ raise PipelexBundleUnexpectedError(message="No pipes section found in bundle spec")+ if pipe_code not in bundle_blueprint.pipe:+ msg = f"Pipe '{pipe_code}' not found in bundle spec but we recorded a dry run failure for it"+ raise PipelexBundleUnexpectedError(message=msg)++ pipe_spec = bundle_blueprint.pipe[pipe_code]+ spec_class = pipe_type_to_spec_class.get(pipe_spec.type)+ if not spec_class:+ msg = f"Unknown pipe type: {pipe_spec.type}"+ raise ValidateDryRunError(msg)+ pipe_spec = spec_class(**pipe_spec.model_dump(serialize_as_any=True))+ pipe_failure = PipeFailure(+ pipe_spec=pipe_spec,+ error_message=dry_run_output.error_message or "",+ )+ dry_run_pipe_failures.append(pipe_failure)+ return dry_run_pipe_failures+++async def dry_run_bundle_blueprint(bundle_blueprint: PipelexBundleBlueprint) -> dict[str, DryRunOutput]:+ library_manager = get_library_manager()+ try:+ pipes = library_manager.load_from_blueprint(blueprint=bundle_blueprint)+ dry_run_result = await dry_run_pipes(pipes=pipes, raise_on_failure=True)+ except StaticValidationError as static_validation_error:+ static_validation_error_data = StaticValidationErrorData(+ error_type=static_validation_error.error_type,+ domain=static_validation_error.domain,+ pipe_code=static_validation_error.pipe_code,+ variable_names=static_validation_error.variable_names,+ required_concept_codes=static_validation_error.required_concept_codes,+ provided_concept_code=static_validation_error.provided_concept_code,+ file_path=static_validation_error.file_path,+ explanation=static_validation_error.explanation,+ )+ raise PipelexBundleError(+ message=static_validation_error.desc(), static_validation_error=static_validation_error_data+ ) from static_validation_error+ except DomainLoadingError as domain_loading_error:+ domain_failures = [DomainFailure(domain_code=domain_loading_error.domain_code, error_message=str(domain_loading_error))]+ raise PipelexBundleError(message=domain_loading_error.message, domain_failures=domain_failures) from domain_loading_error+ except ConceptLoadingError as concept_loading_error:+ concept_def_error = concept_loading_error.concept_definition_error+ concept_definition_error_data = ConceptDefinitionErrorData(+ message=str(concept_def_error),+ domain_code=concept_def_error.domain_code,+ concept_code=concept_def_error.concept_code,+ description=concept_def_error.description,+ structure_class_python_code=concept_def_error.structure_class_python_code,+ source=concept_def_error.source,+ )+ raise PipelexBundleError(+ message=concept_loading_error.message, concept_definition_errors=[concept_definition_error_data]+ ) from concept_loading_error+ except PipeLoadingError as pipe_loading_error:+ pipe_def_error = pipe_loading_error.pipe_definition_error+ pipe_definition_error_data = PipeDefinitionErrorData(+ message=str(pipe_def_error),+ domain_code=pipe_def_error.domain_code,+ pipe_code=pipe_def_error.pipe_code,+ description=pipe_def_error.description,+ source=pipe_def_error.source,+ )+ raise PipelexBundleError(message=pipe_loading_error.message, pipe_definition_errors=[pipe_definition_error_data]) from pipe_loading_error++ return dry_run_resultdiff --git a/pipelex/builder/bundle_header_spec.py b/pipelex/builder/bundle_header_spec.pynew file mode 100644index 000000000..47f8ddeb7--- /dev/null+++ b/pipelex/builder/bundle_header_spec.py@@ -0,0 +1,10 @@+from pydantic import Field++from pipelex.core.stuffs.structured_content import StructuredContent+++class BundleHeaderSpec(StructuredContent):+ domain: str = Field(description="Name of the domain of the knowledge work.")+ description: str = Field(description="Definition of the domain of the knowledge work.")+ system_prompt: str | None = Field(description="System prompt for the domain.")+ main_pipe: str = Field(description="The main pipe of the domain.")diff --git a/pipelex/builder/bundle_spec.py b/pipelex/builder/bundle_spec.pynew file mode 100644index 000000000..1b1958648--- /dev/null+++ b/pipelex/builder/bundle_spec.py@@ -0,0 +1,120 @@+from pydantic import ConfigDict, Field, ValidationError, field_validator, model_validator++from pipelex.builder.builder_errors import (+ ConceptFailure,+ ConceptSpecError,+ PipeBuilderError,+ PipeFailure,+ PipelexBundleError,+ PipeSpecError,+)+from pipelex.builder.concept.concept_spec import ConceptSpec+from pipelex.builder.pipe.pipe_spec_union import PipeSpecUnion+from pipelex.core.bundles.pipe_sorter import sort_pipes_by_dependencies+from pipelex.core.bundles.pipelex_bundle_blueprint import PipeBlueprintUnion, PipelexBundleBlueprint+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error+++class PipelexBundleSpec(StructuredContent):+ """Complete spec of a Pipelex bundle TOML definition.++ Represents the top-level structure of a Pipelex bundle, which defines a domain+ with its concepts, pipes, and configuration. Bundles are the primary unit of+ organization for Pipelex workflows, loaded from TOML files.++ Attributes:+ domain: The domain identifier for this bundle in snake_case format.+ Serves as the namespace for all concepts and pipes within.+ description: Natural language description of the pipeline's purpose and functionality.+ system_prompt: Default system prompt applied to all LLM pipes in the bundle+ unless overridden at the pipe level.+ main_pipe: The main pipe of the bundle.+ concept: Dictionary of concept definitions used in this domain. Keys are concept+ codes in PascalCase format, values are ConceptBlueprint instances or+ string references to existing concepts.+ pipe: Dictionary of pipe definitions for data transformation. Keys are pipe+ codes in snake_case format, values are specific pipe spec types+ (PipeLLM, PipeImgGen, PipeSequence, etc.).++ Validation Rules:+ 1. Domain must be in valid snake_case format.+ 2. Concept keys must be in PascalCase format.+ 3. Pipe keys must be in snake_case format.+ 4. Extra fields are forbidden (strict mode).+ 5. Pipe types must match their blueprint discriminator.++ """++ model_config = ConfigDict(extra="forbid")++ domain: str+ description: str | None = None+ system_prompt: str | None = None+ main_pipe: str++ concept: dict[str, ConceptSpec | str] | None = Field(default_factory=dict)++ pipe: dict[str, PipeSpecUnion] | None = Field(default_factory=dict)++ @field_validator("domain", mode="before")+ @classmethod+ def validate_domain_syntax(cls, domain: str) -> str:+ DomainBlueprint.validate_domain_code(code=domain)+ return domain++ @model_validator(mode="after")+ def validate_main_pipe(self) -> "PipelexBundleSpec":+ if not self.pipe or (self.main_pipe not in self.pipe):+ msg = f"Main pipe '{self.main_pipe}' could not be found in bundle spec"+ raise PipelexBundleError(message=msg)+ return self++ def to_blueprint(self) -> PipelexBundleBlueprint:+ concept: dict[str, ConceptBlueprint | str] | None = None++ if self.concept:+ concept = {}+ for concept_code, concept_spec_or_name in self.concept.items():+ if isinstance(concept_spec_or_name, ConceptSpec):+ try:+ concept[concept_code] = concept_spec_or_name.to_blueprint()+ except ValidationError as exc:+ msg = f"Failed to create concept blueprint from spec for concept code {concept_code}: {format_pydantic_validation_error(exc)}"+ concept_failure = ConceptFailure(concept_spec=concept_spec_or_name, error_message=msg)+ raise ConceptSpecError(message=msg, concept_failure=concept_failure) from exc+ else:+ concept[concept_code] = ConceptBlueprint(description=concept_code, structure=concept_spec_or_name)++ pipe: dict[str, PipeBlueprintUnion] | None = None+ if self.pipe:+ # First, convert all specs to blueprints+ pipe_blueprints: dict[str, PipeBlueprintUnion] = {}+ for pipe_code, pipe_spec in self.pipe.items():+ try:+ pipe_blueprints[pipe_code] = pipe_spec.to_blueprint()+ except ValidationError as exc:+ msg = f"Failed to create pipe blueprint from spec for pipe code {pipe_code}: {format_pydantic_validation_error(exc)}"+ pipe_failure = PipeFailure(pipe_spec=pipe_spec, error_message=msg)+ raise PipeSpecError(message=msg, pipe_failure=pipe_failure) from exc++ # Then, sort blueprints by dependencies+ try:+ sorted_pipe_items = sort_pipes_by_dependencies(pipe_blueprints)+ except Exception as exc:+ msg = f"Failed to sort pipes by dependencies: {exc}"+ raise PipeBuilderError(msg) from exc++ # Finally, create the ordered dict+ pipe = dict(sorted_pipe_items)++ return PipelexBundleBlueprint(+ domain=self.domain,+ description=self.description,+ system_prompt=self.system_prompt,+ main_pipe=self.main_pipe,+ pipe=pipe,+ concept=concept,+ )diff --git a/pipelex/builder/pipe/pipe_design.plx b/pipelex/builder/pipe/pipe_design.plxindex 83754ff1a..fb47efdf5 100644--- a/pipelex/builder/pipe/pipe_design.plx+++ b/pipelex/builder/pipe/pipe_design.plx@@ -116,7 +116,7 @@ This PipeLLM is part of a larger pipeline:You will specifically generate the PipeLLM related to this signature:@pipe_signature-If it's a structured generation, indicate it in the system_prompt to clarify the task.+If we are generating a structured concept, indicate it in the system_prompt to clarify the task. But DO NOT detail the structure in any of the user/system prompts: we will add the schema later. So, don't write a bullet-list of all the attributes to determine.If it's to generate free form text, the prompt should indicate to be concise.If it's to generate an image generation prompt, the prompt should indicate to be VERY concise and focus and apply the best practice for image generation.diff --git a/pipelex/builder/pipe/pipe_llm_spec.py b/pipelex/builder/pipe/pipe_llm_spec.pyindex 600667baf..3c93c1ea0 100644--- a/pipelex/builder/pipe/pipe_llm_spec.py+++ b/pipelex/builder/pipe/pipe_llm_spec.py@@ -87,13 +87,22 @@ class PipeLLMSpec(PipeSpec):temperature: float | None = Field(default=None, ge=0, le=1)system_prompt: str | None = Field(default=None, description="A system prompt to guide the LLM's behavior, style and skills. Can be a template.")prompt: str | None = Field(- description=(- "A template for the user prompt. Use `$` prefix for inline variables (e.g., `$topic`) and `@` prefix "- "to insert content as a block with delimiters (e.g., `@extracted_text` --> extracted_text: ```\n[the extracted_text goes here]\n```). "- "**Notes**: • Image variables must be inserted too. They can be simply added with the `$` prefix on a line, e.g. `$image_1`."- "Or you can mention them by their number in order in the inputs section, starting from 1. Example: "- "`Only analyze the colors from $image_1 and the shapes from $image_2."- )+ description="""A template for the user prompt:+Use `$` prefix for inline variables (e.g., `$topic`) and `@` prefix to insert content as a block with delimiters+For example, `@extracted_text` will generate this:+extracted_text: ```+[the extracted_text goes here]+```+so you don't need to write the delimiters yourself.++**Notes**:+• Image variables must be inserted too.+They can be simply added with the `$` prefix on a line, e.g. `$image_1`.+Or you can mention them by their number in order in the inputs section, starting from 1.+Example: `Only analyze the colors from $image_1 and the shapes from $image_2.+• If we are generating a structured concept, DO NOT detail the structure in the prompt: we will add the schema later.+So, don't have to write a bullet-list of all the attributes definitions yourself.+""")nb_output: int | None = Field(diff --git a/pipelex/builder/pipe/pipe_spec_map.py b/pipelex/builder/pipe/pipe_spec_map.pynew file mode 100644index 000000000..356ed6b60--- /dev/null+++ b/pipelex/builder/pipe/pipe_spec_map.py@@ -0,0 +1,21 @@+from pipelex.builder.pipe.pipe_batch_spec import PipeBatchSpec+from pipelex.builder.pipe.pipe_compose_spec import PipeComposeSpec+from pipelex.builder.pipe.pipe_condition_spec import PipeConditionSpec+from pipelex.builder.pipe.pipe_extract_spec import PipeExtractSpec+from pipelex.builder.pipe.pipe_func_spec import PipeFuncSpec+from pipelex.builder.pipe.pipe_img_gen_spec import PipeImgGenSpec+from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec+from pipelex.builder.pipe.pipe_parallel_spec import PipeParallelSpec+from pipelex.builder.pipe.pipe_sequence_spec import PipeSequenceSpec++pipe_type_to_spec_class: dict[str, type] = {+ "PipeFunc": PipeFuncSpec,+ "PipeImgGen": PipeImgGenSpec,+ "PipeCompose": PipeComposeSpec,+ "PipeLLM": PipeLLMSpec,+ "PipeExtract": PipeExtractSpec,+ "PipeBatch": PipeBatchSpec,+ "PipeCondition": PipeConditionSpec,+ "PipeParallel": PipeParallelSpec,+ "PipeSequence": PipeSequenceSpec,+}diff --git a/pipelex/builder/pipe/pipe_spec_union.py b/pipelex/builder/pipe/pipe_spec_union.pynew file mode 100644index 000000000..d6ffb6f27--- /dev/null+++ b/pipelex/builder/pipe/pipe_spec_union.py@@ -0,0 +1,26 @@+from typing import Annotated++from pydantic import Field++from pipelex.builder.pipe.pipe_batch_spec import PipeBatchSpec+from pipelex.builder.pipe.pipe_compose_spec import PipeComposeSpec+from pipelex.builder.pipe.pipe_condition_spec import PipeConditionSpec+from pipelex.builder.pipe.pipe_extract_spec import PipeExtractSpec+from pipelex.builder.pipe.pipe_func_spec import PipeFuncSpec+from pipelex.builder.pipe.pipe_img_gen_spec import PipeImgGenSpec+from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec+from pipelex.builder.pipe.pipe_parallel_spec import PipeParallelSpec+from pipelex.builder.pipe.pipe_sequence_spec import PipeSequenceSpec++PipeSpecUnion = Annotated[+ PipeFuncSpec+ | PipeImgGenSpec+ | PipeComposeSpec+ | PipeLLMSpec+ | PipeExtractSpec+ | PipeBatchSpec+ | PipeConditionSpec+ | PipeParallelSpec+ | PipeSequenceSpec,+ Field(discriminator="type"),+]diff --git a/pipelex/cli/_cli.py b/pipelex/cli/_cli.pyindex 6382ea9e4..ce224dfcd 100644--- a/pipelex/cli/_cli.py+++ b/pipelex/cli/_cli.py@@ -3,7 +3,7 @@ from click import Command, Contextfrom typer.core import TyperGroupfrom typing_extensions import override-from pipelex.cli.commands import init_app, show_app, validate_app+from pipelex.cli.commands import init_app, run_cmd, show_app, validate_appfrom pipelex.cli.commands.build_cmd import build_appfrom pipelex.cli.commands.kit_cmd import kit_app@@ -37,3 +37,4 @@ app.add_typer(validate_app, name="validate", help="Validation and dry-run commanapp.add_typer(show_app, name="show", help="Show and list commands")app.add_typer(build_app, name="build", help="Build artifacts like pipeline blueprints")app.add_typer(kit_app, name="kit", help="Manage kit assets")+app.command(name="run", help="Execute a pipeline")(run_cmd)diff --git a/pipelex/cli/commands/__init__.py b/pipelex/cli/commands/__init__.pyindex 2a74b504f..e92b3228a 100644--- a/pipelex/cli/commands/__init__.py+++ b/pipelex/cli/commands/__init__.py@@ -4,7 +4,8 @@ This package organizes CLI commands into logical modules."""from pipelex.cli.commands.init_cmd import init_app+from pipelex.cli.commands.run_cmd import run_cmdfrom pipelex.cli.commands.show_cmd import show_appfrom pipelex.cli.commands.validate_cmd import validate_app-__all__ = ["init_app", "show_app", "validate_app"]+__all__ = ["init_app", "run_cmd", "show_app", "validate_app"]diff --git a/pipelex/cli/commands/build_cmd.py b/pipelex/cli/commands/build_cmd.pyindex 5a398ddef..32574e198 100644--- a/pipelex/cli/commands/build_cmd.py+++ b/pipelex/cli/commands/build_cmd.py@@ -18,6 +18,7 @@ build_app = typer.Typer(help="Build working pipelines from natural language requ"""Today's example:+pipelex build pipe "Imagine a cute animal mascot for a startup based on its elevator pitch"pipelex build pipe "Given an expense report, apply company rules"pipelex build pipe "Take a CV in a PDF file, a Job offer text, and analyze if they match"diff --git a/pipelex/cli/commands/run_cmd.py b/pipelex/cli/commands/run_cmd.pynew file mode 100644index 000000000..9278ea1ce--- /dev/null+++ b/pipelex/cli/commands/run_cmd.py@@ -0,0 +1,157 @@+from __future__ import annotations++import asyncio+from typing import Annotated++import typer++from pipelex import log, pretty_print_md+from pipelex.builder.builder import load_pipe_from_bundle+from pipelex.builder.builder_errors import PipelexBundleError+from pipelex.exceptions import PipeInputError+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execute import execute_pipeline+from pipelex.tools.misc.file_utils import get_incremental_file_path+from pipelex.tools.misc.json_utils import JsonTypeError, load_json_dict_from_path, save_as_json_to_path+++def run_cmd(+ target: Annotated[+ str | None,+ typer.Argument(help="Pipe code or bundle file path (auto-detected)"),+ ] = None,+ pipe: Annotated[+ str | None,+ typer.Option("--pipe", help="Explicitly specify pipe code to run"),+ ] = None,+ bundle: Annotated[+ str | None,+ typer.Option("--bundle", help="Bundle file path (.plx) - runs its main_pipe"),+ ] = None,+ inputs: Annotated[+ str | None,+ typer.Option("--inputs", "-i", help="Path to JSON file with input_memory"),+ ] = None,+ output: Annotated[+ str | None,+ typer.Option("--output", "-o", help="Path to save output JSON (default: {pipe_code}.json)"),+ ] = None,+ no_output: Annotated[+ bool,+ typer.Option("--no-output", help="Skip saving output to file"),+ ] = False,+ no_pretty_print: Annotated[+ bool,+ typer.Option("--no-pretty-print", help="Skip pretty printing the main_stuff"),+ ] = False,+) -> None:+ """Execute a pipeline by pipe code or bundle file.++ Examples:+ pipelex run my_pipe+ pipelex run --pipe my_pipe --inputs data.json+ pipelex run --bundle my_bundle.plx+ pipelex run my_bundle.plx --inputs data.json+ pipelex run my_pipe --output results.json --no-pretty-print+ """+ # Initialize Pipelex+ Pipelex.make()++ # Validate mutual exclusivity+ provided_options = sum([target is not None, pipe is not None, bundle is not None])+ if provided_options == 0:+ typer.secho("Failed to run: must provide a pipe code, bundle file, or target", fg=typer.colors.RED, err=True)+ raise typer.Exit(1)+ if provided_options > 1:+ typer.secho(+ "Failed to run: cannot use multiple options (--pipe, --bundle, or positional target) simultaneously",+ fg=typer.colors.RED,+ err=True,+ )+ raise typer.Exit(1)++ async def run_pipeline():+ # Initialize to satisfy linter (will be assigned before use)+ pipe_code: str = ""+ source_description: str = ""+ bundle_path: str | None = None++ # Determine source: bundle path or pipe code+ if bundle:+ bundle_path = bundle+ elif pipe:+ pipe_code = pipe+ source_description = f"pipe '{pipe_code}'"+ elif target:+ if target.endswith(".plx"):+ bundle_path = target+ else:+ pipe_code = target+ source_description = f"pipe '{pipe_code}'"+ else:+ # Should never reach here due to validation above+ typer.secho("Failed to run: no pipe or bundle specified", fg=typer.colors.RED, err=True)+ raise typer.Exit(1)++ # Load bundle if needed+ if bundle_path:+ try:+ pipe_code = await load_pipe_from_bundle(bundle_path)+ source_description = f"bundle '{bundle_path}' • main_pipe: '{pipe_code}'"+ except FileNotFoundError as exc:+ typer.secho(f"Failed to load bundle '{bundle_path}': {exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc+ except PipelexBundleError as exc:+ typer.secho(f"Failed to load bundle '{bundle_path}': {exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc+ except PipeInputError as exc:+ typer.secho(f"Failed to load bundle '{bundle_path}': {exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc++ try:+ # Load inputs if provided+ input_memory = None+ if inputs:+ try:+ input_memory = load_json_dict_from_path(inputs)+ typer.echo(f"Loaded inputs from: {inputs}")+ except FileNotFoundError as file_not_found_exc:+ typer.secho(f"Failed to load input file '{inputs}': file not found", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from file_not_found_exc+ except JsonTypeError as json_type_error_exc:+ typer.secho(f"Failed to parse input file '{inputs}': must be a valid JSON dictionary", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from json_type_error_exc++ # Execute pipeline+ typer.secho(f"\n🚀 Executing {source_description}...\n", fg=typer.colors.GREEN, bold=True)++ pipe_output = await execute_pipeline(+ pipe_code=pipe_code,+ input_memory=input_memory,+ )++ # Pretty print main_stuff unless disabled+ if not no_pretty_print:+ typer.echo("")+ pretty_print_md(content=pipe_output.main_stuff.content.rendered_markdown(), title=f"Main output of '{pipe_code}'")+ typer.echo("")++ # Save working memory to JSON unless disabled+ if not no_output:+ output_path = output or get_incremental_file_path(+ base_path="results",+ base_name=f"run_{pipe_code}",+ extension="json",+ )+ working_memory_dict = pipe_output.working_memory.model_dump()+ save_as_json_to_path(object_to_save=working_memory_dict, path=output_path)+ typer.echo(typer.style(f"✅ Working memory saved to: {output_path}", fg=typer.colors.GREEN))++ typer.echo(typer.style("✅ Pipeline execution completed successfully", fg=typer.colors.GREEN))++ except Exception as exc:+ log.error(f"Error executing pipeline: {exc}")+ typer.secho(f"Failed to execute pipeline: {exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc++ asyncio.run(run_pipeline())diff --git a/pipelex/cli/commands/show_cmd.py b/pipelex/cli/commands/show_cmd.pyindex 9584761e0..43cc0a99f 100644--- a/pipelex/cli/commands/show_cmd.py+++ b/pipelex/cli/commands/show_cmd.py@@ -17,10 +17,7 @@ def do_show_config() -> None:"""Show the pipelex configuration."""try:final_config = config_manager.load_config()- pretty_print(- final_config,- title=f"Pipelex configuration for project: {config_manager.get_project_name()}",- )+ pretty_print(final_config, title="Pipelex configuration")except Exception as exc:msg = f"Error loading configuration: {exc}"raise PipelexConfigError(msg) from excdiff --git a/pipelex/cogt/content_generation/content_generator.py b/pipelex/cogt/content_generation/content_generator.pyindex 91d5b66b4..90a4f555e 100644--- a/pipelex/cogt/content_generation/content_generator.py+++ b/pipelex/cogt/content_generation/content_generator.py@@ -99,7 +99,7 @@ class ContentGenerator(ContentGeneratorProtocol):llm_assignment_factory_to_object = LLMAssignmentFactory(job_metadata=job_metadata,llm_setting=llm_setting_for_object,- llm_prompt_factory=llm_prompt_factory_for_object or LLMPromptTemplate.for_structure_from_preliminary_text(),+ llm_prompt_factory=llm_prompt_factory_for_object or LLMPromptTemplate.make_for_structuring_from_preliminary_text(),)workflow_arg = TextThenObjectAssignment(@@ -169,7 +169,7 @@ class ContentGenerator(ContentGeneratorProtocol):llm_assignment_factory_to_object = LLMAssignmentFactory(job_metadata=job_metadata,llm_setting=llm_setting_for_object_list,- llm_prompt_factory=llm_prompt_factory_for_object_list or LLMPromptTemplate.for_structure_from_preliminary_text(),+ llm_prompt_factory=llm_prompt_factory_for_object_list or LLMPromptTemplate.make_for_structuring_from_preliminary_text(),)workflow_arg = TextThenObjectAssignment(object_class_name=object_class.__name__,diff --git a/pipelex/cogt/llm/llm_prompt_template.py b/pipelex/cogt/llm/llm_prompt_template.pyindex 450b3fdf1..fb34e8d59 100644--- a/pipelex/cogt/llm/llm_prompt_template.py+++ b/pipelex/cogt/llm/llm_prompt_template.py@@ -113,7 +113,7 @@ class LLMPromptTemplate(LLMPromptFactoryAbstract):return llm_prompt@classmethod- def for_structure_from_preliminary_text(cls) -> "LLMPromptTemplate":+ def make_for_structuring_from_preliminary_text(cls) -> "LLMPromptTemplate":llm_config = get_config().cogt.llm_configproto_prompt = LLMPrompt(system_text=llm_config.get_template("structure_from_preliminary_text_system"),diff --git a/pipelex/core/bundles/pipelex_bundle_blueprint.py b/pipelex/core/bundles/pipelex_bundle_blueprint.pyindex c0a8af155..f55c599db 100644--- a/pipelex/core/bundles/pipelex_bundle_blueprint.py+++ b/pipelex/core/bundles/pipelex_bundle_blueprint.py@@ -1,7 +1,8 @@from typing import Annotated-from pydantic import BaseModel, ConfigDict, Field, field_validator+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator+from pipelex.builder.builder_errors import PipelexBundleErrorfrom pipelex.core.concepts.concept_blueprint import ConceptBlueprintfrom pipelex.core.domains.domain_blueprint import DomainBlueprintfrom pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprint@@ -35,8 +36,7 @@ class PipelexBundleBlueprint(BaseModel):domain: strdescription: str | None = Nonesystem_prompt: str | None = None- system_prompt_to_structure: str | None = None- prompt_template_to_structure: str | None = None+ main_pipe: str | None = Noneconcept: dict[str, ConceptBlueprint | str] | None = Field(default_factory=dict)@@ -47,3 +47,10 @@ class PipelexBundleBlueprint(BaseModel):def validate_domain_syntax(cls, domain: str) -> str:DomainBlueprint.validate_domain_code(code=domain)return domain++ @model_validator(mode="after")+ def validate_main_pipe(self) -> "PipelexBundleBlueprint":+ if self.main_pipe and (not self.pipe or (self.main_pipe not in self.pipe)):+ msg = f"Main pipe '{self.main_pipe}' could not be found in bundle spec"+ raise PipelexBundleError(message=msg)+ return selfdiff --git a/pipelex/core/domains/domain.py b/pipelex/core/domains/domain.pyindex 18d53dd51..66fa74e8d 100644--- a/pipelex/core/domains/domain.py+++ b/pipelex/core/domains/domain.py@@ -25,8 +25,6 @@ class Domain(BaseModel):code: strdescription: str | None = Nonesystem_prompt: str | None = None- system_prompt_to_structure: str | None = None- prompt_template_to_structure: str | None = None@classmethoddef make_default(cls) -> Self:diff --git a/pipelex/core/domains/domain_blueprint.py b/pipelex/core/domains/domain_blueprint.pyindex 349b22e3c..9ec0f637b 100644--- a/pipelex/core/domains/domain_blueprint.py+++ b/pipelex/core/domains/domain_blueprint.py@@ -10,8 +10,7 @@ class DomainBlueprint(BaseModel):code: strdescription: strsystem_prompt: str | None = None- system_prompt_to_structure: str | None = None- prompt_template_to_structure: str | None = None+ main_pipe: str | None = None@staticmethoddef validate_domain_code(code: str) -> None:diff --git a/pipelex/core/domains/domain_factory.py b/pipelex/core/domains/domain_factory.pyindex fa8f69353..bab5839a7 100644--- a/pipelex/core/domains/domain_factory.py+++ b/pipelex/core/domains/domain_factory.py@@ -14,8 +14,6 @@ class DomainFactory:code=blueprint.code,description=blueprint.description,system_prompt=blueprint.system_prompt,- system_prompt_to_structure=blueprint.system_prompt_to_structure,- prompt_template_to_structure=blueprint.prompt_template_to_structure,)except ValidationError as exc:validation_error_msg = format_pydantic_validation_error(exc)diff --git a/pipelex/core/interpreter.py b/pipelex/core/interpreter.pyindex d85693850..725349d58 100644--- a/pipelex/core/interpreter.py+++ b/pipelex/core/interpreter.py@@ -28,6 +28,7 @@ class PipelexInterpreter(BaseModel):raise PipelexConfigurationError(msg)return self+ # TODO: rethink this method@staticmethoddef is_pipelex_file(file_path: Path) -> bool:"""Check if a file is a valid Pipelex PLX file.diff --git a/pipelex/hub.py b/pipelex/hub.pyindex 53321d316..b5f7d024d 100644--- a/pipelex/hub.py+++ b/pipelex/hub.py@@ -96,12 +96,11 @@ class PipelexHub:# config (Config): The configuration instance to set."""config = config_manager.load_config(specific_config_path)- config["project_name"] = config_manager.get_project_name()self.set_config(config=config_cls.model_validate(config))def set_config(self, config: ConfigRoot):if self._config is not None:- log.warning(f"set_config() got called but {self._config.project_name} config has already been set")+ log.warning("set_config() got called but it has already been set")returnself._config = configdiff --git a/pipelex/libraries/library_manager.py b/pipelex/libraries/library_manager.pyindex 8a55f93f1..cc7d844a1 100644--- a/pipelex/libraries/library_manager.py+++ b/pipelex/libraries/library_manager.py@@ -61,8 +61,6 @@ class LibraryManager(LibraryManagerAbstract):"domain","description","system_prompt",- "system_prompt_jto_structure",- "prompt_template_to_structure",]def __init__(@@ -187,8 +185,7 @@ class LibraryManager(LibraryManagerAbstract):code=blueprint.domain,description=blueprint.description or "",system_prompt=blueprint.system_prompt,- system_prompt_to_structure=blueprint.system_prompt_to_structure,- prompt_template_to_structure=blueprint.prompt_template_to_structure,+ main_pipe=blueprint.main_pipe,),)diff --git a/pipelex/pipe_operators/llm/pipe_llm.py b/pipelex/pipe_operators/llm/pipe_llm.pyindex 2a5280f5f..cf36e5a69 100644--- a/pipelex/pipe_operators/llm/pipe_llm.py+++ b/pipelex/pipe_operators/llm/pipe_llm.py@@ -15,7 +15,7 @@ from pipelex.cogt.templating.template_category import TemplateCategoryfrom pipelex.config import StaticValidationReaction, get_configfrom pipelex.core.concepts.concept_factory import ConceptFactoryfrom pipelex.core.concepts.concept_native import NativeConceptCode-from pipelex.core.domains.domain import Domain, SpecialDomain+from pipelex.core.domains.domain import SpecialDomainfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.input_requirements import InputRequirementsfrom pipelex.core.pipes.input_requirements_factory import InputRequirementsFactory@@ -35,10 +35,7 @@ from pipelex.hub import (get_content_generator,get_model_deck,get_native_concept,- get_optional_pipe,get_required_concept,- get_required_domain,- get_required_pipe,)from pipelex.pipe_operators.llm.llm_prompt_blueprint import LLMPromptBlueprintfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import StructuringMethod@@ -63,8 +60,6 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):llm_prompt_spec: LLMPromptBlueprintllm_choices: LLMSettingChoices | None = Nonestructuring_method: StructuringMethod | None = None- prompt_template_to_structure: str | None = None- system_prompt_to_structure: str | None = Noneoutput_multiplicity: PipeOutputMultiplicity | None = None@model_validator(mode="after")@@ -84,13 +79,8 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):@overridedef validate_with_libraries(self):- llm_config = get_config().cogt.llm_configself.validate_inputs()self.llm_prompt_spec.validate_with_libraries()- if self.prompt_template_to_structure:- llm_config.get_template(template_name=self.prompt_template_to_structure)- if self.system_prompt_to_structure:- llm_config.get_template(template_name=self.system_prompt_to_structure)if self.llm_choices:for llm_choice in self.llm_choices.list_choices():check_llm_choice_with_deck(llm_choice=llm_choice)@@ -289,42 +279,10 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):llm_prompt_2_factory = Nonecase StructuringMethod.PRELIMINARY_TEXT:log.verbose(f"Creating llm_prompt_2_factory for pipe {self.code} with structuring_method {structuring_method}")- pipe = get_required_pipe(pipe_code=self.code)- # TODO: run_pipe() could get the domain at the same time as the pip_code- domain = get_required_domain(domain=pipe.domain)- prompt_template_to_structure = (- self.prompt_template_to_structure- or domain.prompt_template_to_structure- or llm_config.get_template(template_name="structure_from_preliminary_text_user")- )- system_prompt = self.system_prompt_to_structure or domain.system_prompt- llm_prompt_2_proto = LLMPrompt(- system_text=system_prompt,- user_text=prompt_template_to_structure,- )- llm_prompt_2_factory = LLMPromptTemplate(- proto_prompt=llm_prompt_2_proto,- )+ llm_prompt_2_factory = LLMPromptTemplate.make_for_structuring_from_preliminary_text()elif get_config().pipelex.structure_config.is_default_text_then_structure:log.debug(f"PipeLLM pipe_code is '{self.code}' and is_default_text_then_structure")- # TODO: run_pipe() should get the domain along with the pip_code- if the_pipe := get_optional_pipe(pipe_code=self.code):- domain = get_required_domain(domain=the_pipe.domain)- else:- domain = Domain.make_default()- prompt_template_to_structure = (- self.prompt_template_to_structure- or domain.prompt_template_to_structure- or llm_config.get_template(template_name="structure_from_preliminary_text_user")- )- system_prompt = self.system_prompt_to_structure or domain.system_prompt- llm_prompt_2_proto = LLMPrompt(- system_text=system_prompt,- user_text=prompt_template_to_structure,- )- llm_prompt_2_factory = LLMPromptTemplate(- proto_prompt=llm_prompt_2_proto,- )+ llm_prompt_2_factory = LLMPromptTemplate.make_for_structuring_from_preliminary_text()log.debug(llm_prompt_2_factory, title="llm_prompt_2_factory")else:llm_prompt_2_factory = Nonediff --git a/pipelex/pipe_operators/llm/pipe_llm_blueprint.py b/pipelex/pipe_operators/llm/pipe_llm_blueprint.pyindex d438e73f4..32369e84e 100644--- a/pipelex/pipe_operators/llm/pipe_llm_blueprint.py+++ b/pipelex/pipe_operators/llm/pipe_llm_blueprint.py@@ -25,8 +25,6 @@ class PipeLLMBlueprint(PipeBlueprint):prompt: str | None = Nonestructuring_method: StructuringMethod | None = None- prompt_template_to_structure: str | None = None- system_prompt_to_structure: str | None = Nonenb_output: int | None = Nonemultiple_output: bool | None = Nonediff --git a/pipelex/pipe_operators/llm/pipe_llm_factory.py b/pipelex/pipe_operators/llm/pipe_llm_factory.pyindex c8b277704..7392071ab 100644--- a/pipelex/pipe_operators/llm/pipe_llm_factory.py+++ b/pipelex/pipe_operators/llm/pipe_llm_factory.py@@ -129,7 +129,5 @@ class PipeLLMFactory(PipeFactoryProtocol[PipeLLMBlueprint, PipeLLM]):llm_prompt_spec=llm_prompt_spec,llm_choices=llm_choices,structuring_method=blueprint.structuring_method,- prompt_template_to_structure=blueprint.prompt_template_to_structure,- system_prompt_to_structure=blueprint.system_prompt_to_structure,output_multiplicity=output_multiplicity,)diff --git a/pipelex/pipelex.py b/pipelex/pipelex.pyindex a38b9f575..277169e41 100644--- a/pipelex/pipelex.py+++ b/pipelex/pipelex.py@@ -92,10 +92,7 @@ class Pipelex(metaclass=MetaSingleton):msg = f"Could not setup config because of: {validation_error_msg}"raise PipelexConfigError(msg) from validation_error- log.configure(- project_name=get_config().project_name or "unknown_project",- log_config=get_config().pipelex.log_config,- )+ log.configure(log_config=get_config().pipelex.log_config)log.debug("Logs are configured")# tools@@ -257,12 +254,12 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.self.pipeline_tracker.setup()self.pipeline_manager.setup()- log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} setup done for {get_config().project_name}")+ log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} setup done")def setup_libraries(self):self.library_manager.setup()self.library_manager.load_libraries()- log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} setup libraries done for {get_config().project_name}")+ log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} setup libraries done")def validate_libraries(self):try:@@ -271,7 +268,7 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.validation_error_msg = report_validation_error(category="plx", validation_error=validation_error)msg = f"Could not validate libraries because of: {validation_error_msg}"raise PipelexSetupError(msg) from validation_error- log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} validate libraries done for {get_config().project_name}")+ log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} validate libraries done")def teardown(self):# pipelex@@ -290,7 +287,7 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.self.class_registry.teardown()func_registry.teardown()- log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} teardown done for {get_config().project_name} (except config & logs)")+ log.debug(f"{PACKAGE_NAME} version {PACKAGE_VERSION} teardown done (except config & logs)")self.pipelex_hub.reset_config()# Clear the singleton instance from metaclassif self.__class__ in MetaSingleton.instances:diff --git a/pipelex/system/configuration/config_loader.py b/pipelex/system/configuration/config_loader.pyindex 85380ef09..5b009aac5 100644--- a/pipelex/system/configuration/config_loader.py+++ b/pipelex/system/configuration/config_loader.py@@ -1,6 +1,5 @@import importlibimport os-from configparser import ConfigParserfrom typing import Anyfrom pipelex.system.configuration.config_root import (@@ -160,66 +159,5 @@ class ConfigLoader:return pipelex_config- def get_project_name(self) -> str | None:- """Get the project name from configuration files.-- Checks the following files in order:- 1. pipelex's pyproject.toml- 2. Local pyproject.toml- 3. setup.cfg- 4. setup.py-- Returns:- str | None: The project name or None if not found-- """- # First check pipelex's pyproject.toml- pipelex_pyproject_path = os.path.join(os.path.dirname(os.getcwd()), "pyproject.toml")- if pipelex_pyproject := load_toml_from_path_if_exists(path=pipelex_pyproject_path):- if (project_name := pipelex_pyproject.get("project", {}).get("name")) and isinstance(project_name, str):- return str(project_name)-- # Check local pyproject.toml- local_pyproject_path = os.path.join(os.getcwd(), "pyproject.toml")- if local_pyproject := load_toml_from_path_if_exists(local_pyproject_path):- name_obj: object = local_pyproject.get("project", {}).get("name") or local_pyproject.get("tool", {}).get("poetry", {}).get("name")- if isinstance(name_obj, str):- return name_obj-- # Check setup.cfg- setup_cfg_path = os.path.join(os.getcwd(), "setup.cfg")- try:- config = ConfigParser()- config.read(setup_cfg_path)- if (cfg_name := config.get("metadata", "name", fallback=None)) and config.has_section("metadata"):- return cfg_name- except FileNotFoundError as exc:- print(f"setup.cfg not found at {setup_cfg_path}: {exc}")- except (ValueError, OSError) as exc:- print(f"Failed to parse setup.cfg at {setup_cfg_path}: {exc}")-- # Check setup.py as last resort- setup_py_path = os.path.join(os.getcwd(), "setup.py")- try:- with open(setup_py_path) as f:- content = f.read()- # Simple string search for name parameter- for line in content.splitlines():- if "name=" in line or "name =" in line:- # Extract value between quotes- for quote in ['"', "'"]:- start = line.find(quote)- if start != -1:- end = line.find(quote, start + 1)- if end != -1:- return line[start + 1 : end]- except FileNotFoundError as exc:- print(f"setup.py not found at {setup_py_path}: {exc}")- except (OSError, UnicodeDecodeError) as exc:- print(f"Failed to read setup.py at {setup_py_path}: {exc}")-- print("Could not find project name in any of the configuration files")- return None-config_manager = ConfigLoader()diff --git a/pipelex/system/configuration/config_root.py b/pipelex/system/configuration/config_root.pyindex b10132129..f6b95cac8 100644--- a/pipelex/system/configuration/config_root.py+++ b/pipelex/system/configuration/config_root.py@@ -11,14 +11,7 @@ CONFIG_BASE_OVERRIDES_AFTER_ENV = ["super"]class ConfigRoot(ConfigModel):- """Main configuration class for the project.-- Attributes:- project_name (str): Name of the current project.-- """-- project_name: str | None = None+ """Main configuration class for the project."""def __init__(self, **kwargs: Any):"""Initialize the Config instance.diff --git a/pipelex/tools/log/log.py b/pipelex/tools/log/log.pyindex 32da57727..d3c9d6fd1 100644--- a/pipelex/tools/log/log.py+++ b/pipelex/tools/log/log.py@@ -16,7 +16,6 @@ class Log:def __init__(self):"""Initialize the Log class with default attributes."""- self.project_name: str | None = Noneself._log_config_instance: LogConfig | None = Noneself.rich_handler: logging.Handler | None = Noneself.poor_handler: logging.Handler | None = None@@ -62,21 +61,15 @@ class Log:handler.close()logging.shutdown()- self.project_name = Noneself._log_config_instance = Noneself.rich_handler = Noneself.poor_handler = Noneself.log_dispatch.reset()- def configure(- self,- project_name: str,- log_config: LogConfig,- ):+ def configure(self, log_config: LogConfig):"""Configure the logging system with the given project name and log configuration.Args:- project_name (str): The name of the project.log_config (LogConfig): The log configuration to use.Raises:@@ -87,14 +80,10 @@ class Log:msg = "LogConfig is already set. You can only call log.configure() once."raise RuntimeError(msg)- self.log_dispatch.configure(- project_name=project_name,- log_config=log_config,- )+ self.log_dispatch.configure(log_config=log_config)self._log_config_instance = log_config- self.project_name = project_nameself.rich_handler = log_config.rich_log_config.make_rich_handler()self.rich_handler.setFormatter(EmojiLogFormatter())@@ -114,8 +103,7 @@ class Log:self.set_levels_for_packages(package_log_levels=log_config.package_log_levels)- self.verbose("Logs configured")- self.verbose(f"Config set for {project_name}")+ self.verbose("Logs configured and config set")def set_poor_log_formatter(self, formatter: logging.Formatter):"""Set the formatter for the poor log handler.diff --git a/pipelex/tools/log/log_dispatch.py b/pipelex/tools/log/log_dispatch.pyindex a902afbf6..9822cd507 100644--- a/pipelex/tools/log/log_dispatch.py+++ b/pipelex/tools/log/log_dispatch.py@@ -17,7 +17,6 @@ class LogDispatch:######################################################### TODO: more elegant init for log_dispatch / logdef __init__(self):- self.project_name: str | None = Noneself._log_config_instance: LogConfig | None = Noneself.log_mode: LogMode = LogMode.RICH@@ -26,7 +25,6 @@ class LogDispatch:def reset(self):"""Reset the log dispatch."""- self.project_name = Noneself._log_config_instance = None@property@@ -45,15 +43,10 @@ class LogDispatch:raise RuntimeError(msg)return self._log_config_instance- def configure(- self,- project_name: str,- log_config: LogConfig,- ):- """Configures the LogDispatch with project name and log configuration.+ def configure(self, log_config: LogConfig):+ """Configures the LogDispatch with log configuration.Args:- project_name (str): The name of the project.log_config (LogConfig): The log configuration to use.Raises:@@ -64,7 +57,6 @@ class LogDispatch:msg = "LogConfig is already set. You can only call log.configure() once."raise RuntimeError(msg)self._log_config_instance = log_config- self.project_name = project_nameself.log_mode = log_config.log_mode########################################################@@ -275,10 +267,7 @@ class LogDispatch:if module_file == logging_module_path or module_file.endswith("/log.py"):continueif module.__name__ == "__main__":- if self.project_name is None:- msg = "Project name is not set. You must call initialize Pipelex first."- raise RuntimeError(msg)- log_origin_name = self.project_name+ log_origin_name = "Unknown"else:log_origin_name = module.__name__.split(sep=".", maxsplit=1)[0]breakdiff --git a/pipelex/tools/misc/json_utils.py b/pipelex/tools/misc/json_utils.pyindex 7950dfe84..6b83b5461 100644--- a/pipelex/tools/misc/json_utils.py+++ b/pipelex/tools/misc/json_utils.py@@ -122,10 +122,10 @@ def load_json_dict_from_path(path: str) -> dict[Any, Any]:"""json_content: JsonContent = load_json_from_path(path)- if isinstance(json_content, dict):- return json_content- msg = f"{path} is not a dict"- raise JsonTypeError(msg)+ if not isinstance(json_content, dict):+ msg = f"{path} is not a dict"+ raise JsonTypeError(msg)+ return json_contentdef load_json_list_from_path(path: str) -> list[Any]:diff --git a/tests/conftest.py b/tests/conftest.pyindex 108c4e6a3..0652aa35f 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -24,7 +24,6 @@ def reset_pipelex_config_fixture():config = get_config()log.verbose(config, title="Test config")assert isinstance(config, pipelex.config.PipelexConfig)- assert config.project_name == "pipelex"except Exception as exc:Console().print(Traceback())pytest.exit(f"Critical Pipelex setup error: {exc}")diff --git a/tests/unit/pipelex/core/interpreter/test_interpreter.py b/tests/unit/pipelex/core/interpreter/test_interpreter.pyindex 4690a2164..32073c02a 100644--- a/tests/unit/pipelex/core/interpreter/test_interpreter.py+++ b/tests/unit/pipelex/core/interpreter/test_interpreter.py@@ -15,16 +15,16 @@ class TestPipelexInterpreter:test_file.write_text("domain = 'test'\n[concept]\n")content = "domain = 'other'\n[concept]\n"- converter = PipelexInterpreter(file_path=test_file, file_content=content)- assert converter.file_path == test_file- assert converter.file_content == content+ interpreter = PipelexInterpreter(file_path=test_file, file_content=content)+ assert interpreter.file_path == test_file+ assert interpreter.file_content == content@pytest.mark.parametrize(("test_name", "plx_content", "expected_blueprint"), InterpreterTestCases.VALID_TEST_CASES)def test_make_pipelex_bundle_blueprint(self, test_name: str, plx_content: str, expected_blueprint: PipelexBundleBlueprint):"""Test making blueprint from various valid PLX content."""- converter = PipelexInterpreter(file_content=plx_content)+ interpreter = PipelexInterpreter(file_content=plx_content)- blueprint = converter.make_pipelex_bundle_blueprint()+ blueprint = interpreter.make_pipelex_bundle_blueprint()pretty_print(blueprint, title=f"Blueprint {test_name}")pretty_print(expected_blueprint, title=f"Expected blueprint {test_name}")assert blueprint == expected_blueprint@@ -32,8 +32,8 @@ class TestPipelexInterpreter:@pytest.mark.parametrize(("test_name", "invalid_plx_content", "expected_exception"), InterpreterTestCases.ERROR_TEST_CASES)def test_invalid_plx_should_raise_exception(self, test_name: str, invalid_plx_content: str, expected_exception: type[Exception]):"""Test that invalid PLX content raises appropriate exceptions."""- converter = PipelexInterpreter(file_content=invalid_plx_content)+ interpreter = PipelexInterpreter(file_content=invalid_plx_content)log.verbose(f"Testing invalid PLX content: {test_name}")with pytest.raises(expected_exception):- converter.make_pipelex_bundle_blueprint()+ interpreter.make_pipelex_bundle_blueprint()diff --git a/tests/unit/pipelex/core/test_data/domain/simple_domains.py b/tests/unit/pipelex/core/test_data/domain/simple_domains.pyindex 4367a11dd..4a7bd5c0a 100644--- a/tests/unit/pipelex/core/test_data/domain/simple_domains.py+++ b/tests/unit/pipelex/core/test_data/domain/simple_domains.py@@ -16,15 +16,11 @@ DOMAIN_WITH_SYSTEM_PROMPTS = ("""domain = "system_prompt_test"description = "A domain with system prompts"system_prompt = "You are an expert assistant"-system_prompt_to_structure = "Generate structured output"-prompt_template_to_structure = "Structure the following data: ..."""",PipelexBundleBlueprint(domain="system_prompt_test",description="A domain with system prompts",system_prompt="You are an expert assistant",- system_prompt_to_structure="Generate structured output",- prompt_template_to_structure="Structure the following data: ...",),)diff --git a/tests/unit/pipelex/tools/config/test_config_manager.py b/tests/unit/pipelex/tools/config/test_config_manager.pydeleted file mode 100644index 1a13f1eb5..000000000--- a/tests/unit/pipelex/tools/config/test_config_manager.py+++ /dev/null@@ -1,15 +0,0 @@-from pipelex import pretty_print-from pipelex.config import PipelexConfig-from pipelex.hub import get_pipelex_hub-from pipelex.system.configuration.config_loader import config_manager---class TestConfigManager:- def test_get_project_name(self):- project_name = config_manager.get_project_name()- pretty_print(project_name, title="project_name")- assert project_name == "pipelex"-- def test_load_pipelex_template_config(self):- hub = get_pipelex_hub()- hub.setup_config(config_cls=PipelexConfig, specific_config_path="pipelex/kit/configs/pipelex.toml")diff --git a/tests/unit/pipelex/tools/config/test_load_config.py b/tests/unit/pipelex/tools/config/test_load_config.pynew file mode 100644index 000000000..d0c297161--- /dev/null+++ b/tests/unit/pipelex/tools/config/test_load_config.py@@ -0,0 +1,8 @@+from pipelex.config import PipelexConfig+from pipelex.hub import get_pipelex_hub+++class TestLoadConfig:+ def test_load_config(self):+ hub = get_pipelex_hub()+ hub.setup_config(config_cls=PipelexConfig, specific_config_path="pipelex/kit/configs/pipelex.toml")