Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
fix/docs-sane-lists (#404)
Thomas Hebrardcccf79a48a3c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ea69d0843..2054d7d0a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,6 +1,6 @@# Changelog-## Unreleased+## [v0.14.0] - 2025-10-27### Added- **`pipelex doctor` command**: Diagnoses and fixes common configuration issues including missing files, invalid telemetry settings, and unset environment variables for enabled backends.@@ -19,6 +19,10 @@- **Default enabled backends**: Amazon Bedrock, Google AI, and Google Vertex AI are now enabled by default.- **Naming consistency**: "AWS Bedrock" renamed to "Amazon Bedrock" throughout codebase, configuration, and documentation.+### Fixed++- Some documentation links were broken.+## [v0.13.2] - 2025-10-25### Addeddiff --git a/Makefile b/Makefileindex f0454e7a6..c7684cf2b 100644--- a/Makefile+++ b/Makefile@@ -458,7 +458,7 @@ check-TODOs: envdocs: env$(call PRINT_TITLE,"Serving documentation with mkdocs")- $(VENV_MKDOCS) serve --watch docs+ $(VENV_MKDOCS) serve -a 127.0.0.1:8000 --watch docsdocs-check: env$(call PRINT_TITLE,"Checking documentation build with mkdocs")diff --git a/docs/pages/advanced-customization/observer-provider-injection.md b/docs/pages/advanced-customization/observer-provider-injection.mdindex 3f45bfb5f..f277ef067 100644--- a/docs/pages/advanced-customization/observer-provider-injection.md+++ b/docs/pages/advanced-customization/observer-provider-injection.md@@ -5,6 +5,7 @@ The Observer system in Pipelex allows you to monitor and collect data about pipe## OverviewThe Observer system is designed to collect execution data at three key moments:+- **Before running a pipe**: Capture initial state and inputs- **After successful execution**: Record outputs and success metrics- **After failed execution**: Log errors and failure context@@ -136,6 +137,7 @@ get_pipelex_hub().set_observer(local_observer)```The LocalObserver creates separate JSONL files for each event type:+- `before_run.jsonl` - Pre-execution data- `successful_run.jsonl` - Success events- `failing_run.jsonl` - Failure eventsdiff --git a/docs/pages/api/index.md b/docs/pages/api/index.mdindex ae74539d6..a9497acb4 100644--- a/docs/pages/api/index.md+++ b/docs/pages/api/index.md@@ -8,7 +8,7 @@ This guide covers everything you need to know about using the Pipelex API to exe2. [Execute Pipeline](#execute-pipeline)3. [Input Format: PipelineInputs](#input-format-pipelineinputs)- [Case 1: Direct Content Format](#case-1-direct-content-format)- - [Case 2: Explicit Format](#case-2-explicit-format-concept-content)+ - [Case 2: Explicit Format (Concept and Content)](#case-2-explicit-format-concept-and-content)4. [Search Domains Explained](#search-domains-explained)5. [PLX Content: Execute Pipelines Inline](#plx-content-execute-pipelines-inline)6. [Error Handling](#error-handling)@@ -77,6 +77,7 @@ Execute a Pipelex pipeline with flexible inputs.**Endpoint:** `POST /pipeline/{pipe_code}/execute`**Path Parameters:**+- `pipe_code` (string, required): The code identifying the pipeline to execute**Request Body:**@@ -94,7 +95,8 @@ Execute a Pipelex pipeline with flexible inputs.```**Request Fields:**-- `inputs` (PipelineInputs): Flexible input format - see [Input Format](#input-format-pipelineinputs) below++- `inputs` (PipelineInputs): Flexible input format - see [Input Format: PipelineInputs](#input-format-pipelineinputs) below- `output_name` (string, optional): Name for the output slot- `output_multiplicity` (string, optional): Output multiplicity setting- `dynamic_output_concept_code` (string, optional): Override output concept@@ -135,6 +137,7 @@ The `inputs` field uses **PipelineInputs** format - a smart, flexible way to pro### How Input Formatting Works**Case 1: Direct Content** - Provide the value directly (simplest)+- 1.1: String → `"my text"`- 1.2: List of strings → `["text1", "text2"]`- 1.3: StructuredContent object → `MyClass(arg1="value")`@@ -144,6 +147,7 @@ The `inputs` field uses **PipelineInputs** format - a smart, flexible way to pro**Note:** Cases 1.3 and 1.5 are at the same level - both handle content types that inherit from `StuffContent`, but for different purposes (custom classes vs. list wrappers).**Case 2: Explicit Format** - Use `{"concept": "...", "content": "..."}` for control (plain dict or DictStuff instance)+- 2.1: String with concept → `{"concept": "Text", "content": "my text"}`- 2.2: List of strings with concept → `{"concept": "Text", "content": ["text1", "text2"]}`- 2.3: StructuredContent object with concept → `{"concept": "Invoice", "content": InvoiceObject}`@@ -203,12 +207,14 @@ inputs = {```**What is StructuredContent?**+- `StructuredContent` is the base class for user-defined data structures in Pipelex- You create your own classes by inheriting from `StructuredContent`- These classes are defined in your project's Python files- Learn more: [Python StructuredContent Classes](../build-reliable-ai-workflows-with-pipelex/define_your_concepts.md#3-python-structuredcontent-classes)**Concept Resolution:**+- The system searches all available domains for a concept matching the class name- If multiple concepts with the same name exist in different domains → **Error**: Must specify domain- If no concept is found → **Error**@@ -228,10 +234,12 @@ inputs = {```**What it accepts:**+- Lists of `StructuredContent` objects (user-defined classes)- Lists of native content objects (`TextContent`, `ImageContent`, etc.)**Requirements:**+- All items must be of the same type- Concept resolution follows the same rules as 1.3- Creates a new `ListContent` wrapper internally@@ -253,16 +261,19 @@ inputs = {```**Key Difference from Case 1.4:**+- Case 1.4: Plain Python list `[item1, item2]` → **Creates** a new `ListContent` wrapper- Case 1.5: Already wrapped `ListContent(items=[item1, item2])` → **Uses** the wrapper directly**Why Case 1.5 is Separate from Case 1.3:**+- `StructuredContent` and `ListContent` are **sibling classes** (both inherit from `StuffContent`)- Case 1.3 handles user-defined structured data classes- Case 1.5 handles list container wrappers- They're at the same inheritance level, not parent-child**Requirements:**+- All items within the `ListContent` must be `StuffContent` objects (this includes both `StructuredContent` and native content like `TextContent`, `ImageContent`)- All items must be of the same type- The `ListContent` cannot be empty@@ -272,7 +283,7 @@ inputs = {----## Case 2: Explicit Format (Concept + Content)+## Case 2: Explicit Format (Concept and Content)Use the explicit format `{"concept": "...", "content": "..."}` when you need precise control over concept selection or when working with domain-specific concepts.@@ -290,6 +301,7 @@ Use the explicit format `{"concept": "...", "content": "..."}` when you need pre```**Concept Options:**+- `"Text"` or `"native.Text"` for native text- Any custom concept that is strictly compatible with `native.Text`@@ -328,6 +340,7 @@ Use the explicit format `{"concept": "...", "content": "..."}` when you need pre**Concept Resolution with Search Domains:**When you specify a concept name without a domain prefix:+- ✅ If the concept exists in only one domain → Automatically found- ❌ If the concept exists in multiple domains → **Error**: "Multiple concepts found. Please specify domain as 'domain.Concept'"- ❌ If the concept doesn't exist → **Error**: "Concept not found"@@ -467,10 +480,12 @@ response = await client.execute_pipeline(```**DictStuff Structure:**+- `concept` (str): The concept code (with optional domain prefix)- `content` (dict[str, Any] | list[Any]): The actual data content**Content Types:**+- **Dictionary**: Single structured object → Creates a single Stuff- **List of dicts**: Multiple structured objects → Creates ListContent with validated items- **List of strings** (for Text-compatible concepts): Creates ListContent of TextContent@@ -546,6 +561,7 @@ Combine different input types in a single request:```In this example:+- `text` uses direct string format (Case 1.1)- `category` uses explicit format with structured content (Case 2.5)- `options` uses direct list format (Case 1.2)@@ -560,6 +576,7 @@ The `plx_content` field allows you to execute pipelines by providing their `.plx### When to Use PLX ContentUse `plx_content` when you want to:+- Execute a dynamically generated pipeline- Test a pipeline without deploying it- Run one-off pipelines that don't need to be stored@@ -592,6 +609,7 @@ The `plx_content` should contain a valid `.plx` file as a string:```**Key Points:**+- The `plx_content` must be a valid Pipelex pipeline definition- Include newlines (`\n`) to properly format the pipeline- The pipeline will be validated before execution@@ -743,6 +761,7 @@ When you provide `plx_content`:```3. **Validate Before Sending**+- Ensure all required fields are present- Match field types to concept structure- Use domain prefixes when concepts might be ambiguousdiff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/kick-off-a-pipelex-workflow-project.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/kick-off-a-pipelex-workflow-project.mdindex b070febef..7e9343fe0 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/kick-off-a-pipelex-workflow-project.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/kick-off-a-pipelex-workflow-project.md@@ -31,6 +31,7 @@ Provide a clear and concise answer.```This creates a simple Q&A pipeline with:+- A domain called "tutorial"- Two concepts: Question and Answer- One pipe that transforms a Question into an Answerdiff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeCompose.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeCompose.mdindex 3fac46a2c..b01451c7c 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeCompose.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeCompose.md@@ -61,6 +61,7 @@ extra_context = { report_date = "2023-10-27" }```In this scenario:+- `PipeCompose` will load the `weekly_report.md` template.- It will use the `user` and `activities` objects from the working memory.- It will add `report_date` to the context from the `extra_context` table.diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeExtract.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeExtract.mdindex 1466c0fca..41bec8749 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeExtract.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeExtract.md@@ -31,6 +31,7 @@ PipeExtract uses the unified inference backend system to manage OCR models. This- Route OCR requests to different backends based on your routing profileCommon OCR model handles:+- `mistral-ocr`: Mistral's OCR model for high-quality text and image extraction- `pypdfium2-extract-text`: Local PDF text extraction (no API calls required)diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeFunc.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeFunc.mdindex cedf88c12..f3efde5f4 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeFunc.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeFunc.md@@ -17,16 +17,18 @@ The `PipeFunc` operator provides an essential escape hatch, allowing you to execFor a function to be automatically registered and available to `PipeFunc`, it **must** meet all of the following criteria:!!! warning "Function Eligibility Requirements"- - **Must be decorated with** `@pipe_func()` (required since v0.12.0)- - **Must be an async function** (defined with `async def`)- - **Must have exactly 1 parameter** named `working_memory`- - **Parameter type must be** `WorkingMemory`- - **Return type must be** a subclass of `StuffContent` (or a generic type like `ListContent[SomeType]`)- - **Must be discoverable** (not in excluded directories like `.venv`, `__pycache__`, etc.)++- **Must be decorated with** `@pipe_func()` (required since v0.12.0)+- **Must be an async function** (defined with `async def`)+- **Must have exactly 1 parameter** named `working_memory`+- **Parameter type must be** `WorkingMemory`+- **Return type must be** a subclass of `StuffContent` (or a generic type like `ListContent[SomeType]`)+- **Must be discoverable** (not in excluded directories like `.venv`, `__pycache__`, etc.)### Return valuesYour async Python function can return:+- A `StuffContent` object (e.g., `TextContent`, `ImageContent`, or a custom `StructuredContent` model).- A `ListContent` containing `StuffContent` objects.diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeImgGen.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeImgGen.mdindex c498b51f4..6f8c8282d 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeImgGen.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeImgGen.md@@ -26,6 +26,7 @@ PipeImgGen uses the unified inference backend system to manage image generation- Route image generation requests to different backends based on your routing profileCommon image generation model handles:+- `base-img-gen`: Base image generation model (alias for flux-pro/v1.1)- `best-img-gen`: Best quality image generation model (alias for flux-pro/v1.1-ultra)- `fast-img-gen`: Fast image generation model (alias for fast-lightning-sdxl)diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.mdindex 8777ec3db..5cc4ab036 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/pipe-operators/PipeLLM.md@@ -67,6 +67,7 @@ Focus on typography, spacing, and overall composition.```In this example:+- `page_content` is the input variable containing a `PageContent` object- `page_view` is the `ImageContent` field within the `PageContent` structure- The dot notation `page_content.page_view` tells Pipelex to extract the image from that specific fielddiff --git a/docs/pages/configuration/config-technical/aws-config.md b/docs/pages/configuration/config-technical/aws-config.mdindex 15a7ef5f7..b4c464049 100644--- a/docs/pages/configuration/config-technical/aws-config.md+++ b/docs/pages/configuration/config-technical/aws-config.md@@ -15,6 +15,7 @@ api_key_method = "env" # or "secret_provider"### Environment Variables Method (`"env"`)When using `api_key_method = "env"`, Pipelex expects the following environment variables:+- `AWS_ACCESS_KEY_ID`: Your AWS access key ID- `AWS_SECRET_ACCESS_KEY`: Your AWS secret access key- `AWS_REGION`: Your AWS region@@ -28,11 +29,15 @@ AWS_REGION=us-east-1### Secret Provider Method (`"secret_provider"`)When using `api_key_method = "secret_provider"`, Pipelex will:+1. Connect to your configured secret provider+2. Look for the same keys as environment variables:- - `AWS_ACCESS_KEY_ID`- - `AWS_SECRET_ACCESS_KEY`- - `AWS_REGION`+<ul>+ <li><code>AWS_ACCESS_KEY_ID</code></li>+ <li><code>AWS_SECRET_ACCESS_KEY</code></li>+ <li><code>AWS_REGION</code></li>+</ul>!!! warning "Secret Provider Requirements"To use the secret provider method, you must:diff --git a/docs/pages/configuration/config-technical/inference-backend-config.md b/docs/pages/configuration/config-technical/inference-backend-config.mdindex 449c28022..91010ca42 100644--- a/docs/pages/configuration/config-technical/inference-backend-config.md+++ b/docs/pages/configuration/config-technical/inference-backend-config.md@@ -84,9 +84,9 @@ Pipelex Inference is a unified inference backend that provides access to all maj### Setup1. **Get your API key:**- - Visit [https://go.pipelex.com/discord](https://go.pipelex.com/discord) to join our Discord- - Request your free API key in the appropriate channel- - No credit card required (limited time offer)+- Visit [https://go.pipelex.com/discord](https://go.pipelex.com/discord) to join our Discord+- Request your free API key in the appropriate channel+- No credit card required (limited time offer)2. **Configure environment variables:**```bashdiff --git a/docs/pages/configuration/config-technical/library-config.md b/docs/pages/configuration/config-technical/library-config.mdindex 2aef0687c..d11641427 100644--- a/docs/pages/configuration/config-technical/library-config.md+++ b/docs/pages/configuration/config-technical/library-config.md@@ -56,6 +56,7 @@ your_project/```**Benefits:**+- Related things stay together- Easy to find pipeline for a specific module- Natural code organization@@ -77,6 +78,7 @@ your_project/```**Benefits:**+- All pipelines in one location- Simple structure for small projectsdiff --git a/docs/pages/configuration/index.md b/docs/pages/configuration/index.mdindex 0fcec5b96..d1140c6ee 100644--- a/docs/pages/configuration/index.md+++ b/docs/pages/configuration/index.md@@ -40,6 +40,7 @@ The exact loading sequence is:6. Super user overrides (`pipelex_super.toml`) (recommended to put in .gitignore)Each subsequent configuration file in this sequence can override settings from the previous ones. This means:+- Settings in `pipelex_local.toml` override the base configuration- Environment-specific settings override local settings- Run mode settings override environment settingsdiff --git a/docs/pages/observer/data-extraction.md b/docs/pages/observer/data-extraction.mdindex de859cc51..ccb8f3888 100644--- a/docs/pages/observer/data-extraction.md+++ b/docs/pages/observer/data-extraction.md@@ -26,15 +26,18 @@ The observer extracts data at three critical moments during pipe execution:The data extraction location depends on your observer implementation:### LocalObserver (Default)+- **Location**: Local filesystem in JSONL format- **Default Directory**: Configured in your Pipelex settings under `observer_config.observer_dir`- **Files Created**:- - `before_run.jsonl` - Pre-execution snapshots- - `successful_run.jsonl` - Success events and results- - `failing_run.jsonl` - Failure events and error context++- `before_run.jsonl` - Pre-execution snapshots+- `successful_run.jsonl` - Success events and results+- `failing_run.jsonl` - Failure events and error context### Custom ObserversYou can implement custom data extraction to:+- **Databases**: PostgreSQL, MongoDB, InfluxDB for structured analysis- **Cloud Storage**: S3, GCS for large-scale data retention- **Streaming Systems**: Kafka, Redis for real-time processingdiff --git a/docs/pages/setup/configure-ai-providers.md b/docs/pages/setup/configure-ai-providers.mdindex dae42dfaf..3560576c3 100644--- a/docs/pages/setup/configure-ai-providers.md+++ b/docs/pages/setup/configure-ai-providers.md@@ -18,8 +18,9 @@ Get **free credits** for testing and development with a single API key that work**Setup:**1. Join our Discord community to get your free API key:- - Visit [https://go.pipelex.com/discord](https://go.pipelex.com/discord)- - Request your key in the appropriate channel++- Visit [https://go.pipelex.com/discord](https://go.pipelex.com/discord)+- Request your key in the appropriate channel2. Create a `.env` file in your project root:```env@@ -131,12 +132,15 @@ pipelex init config```This creates a `.pipelex/` directory with:--- `pipelex.toml`: Feature flags, logging, cost reporting-- `inference/`: LLM configuration and model presets- - `backends.toml`: Enable/disable model providers- - `deck/base_deck.toml`: LLM presets and aliases- - `routing_profiles.toml`: Model routing configuration+```+.pipelex/+├── pipelex.toml # Feature flags, logging, cost reporting+└── inference/ # LLM configuration and model presets+ ├── backends.toml # Enable/disable model providers+ ├── deck/+ │ └── base_deck.toml # LLM presets and aliases+ └── routing_profiles.toml # Model routing configuration+```Learn more in our [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) guide.diff --git a/docs/pages/tools/cli.md b/docs/pages/tools/cli.mdindex c6e5ffde8..d97ec6499 100644--- a/docs/pages/tools/cli.md+++ b/docs/pages/tools/cli.md@@ -432,24 +432,28 @@ For complete documentation including all options and examples, see [Kit Commands## Usage Tips1. **Initial Setup**- - Run `pipelex init config` to create configuration files- - Configure your AI providers in `.pipelex/inference/backends.toml`- - Install agent rules with `pipelex kit rules` if using AI assistants++- Run `pipelex init config` to create configuration files+- Configure your AI providers in `.pipelex/inference/backends.toml`+- Install agent rules with `pipelex kit rules` if using AI assistants2. **Development Workflow**- - Write or generate pipelines in `.plx` files- - Validate with `pipelex validate your_pipe_code` or `pipelex validate your_bundle.plx` during development++- Write or generate pipelines in `.plx` files+- Validate with `pipelex validate your_pipe_code` or `pipelex validate your_bundle.plx` during development- Run `pipelex validate all` before committing changes3. **Running Pipelines**- - Use `pipelex show pipes` to see available pipes- - Use `pipelex show pipe pipe_code` to inspect pipe details- - Run with `pipelex run pipe_code`, add the required inputs using `--inputs`++- Use `pipelex show pipes` to see available pipes+- Use `pipelex show pipe pipe_code` to inspect pipe details+- Run with `pipelex run pipe_code`, add the required inputs using `--inputs`4. **Configuration Management**- - Use `pipelex show config` to verify current settings- - Use `pipelex show backends` to check inference backend setup- - Use `pipelex show models backend_name` to see available models++- Use `pipelex show config` to verify current settings+- Use `pipelex show backends` to check inference backend setup+- Use `pipelex show models backend_name` to see available models## Related Documentationdiff --git a/mkdocs.yml b/mkdocs.ymlindex 797964ee9..3385792ef 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -66,6 +66,7 @@ plugins:markdown_extensions:- meta+ - sane_lists- admonition- attr_list- md_in_html@@ -134,6 +135,8 @@ nav:- LLM Structured Generation: pages/build-reliable-ai-workflows-with-pipelex/llm-structured-generation-config.md- LLM Prompting Style: pages/build-reliable-ai-workflows-with-pipelex/adapt-to-llm-prompting-style-openai-anthropic-mistral.md- LLM Providers & Models: pages/configuration/config-technical/inference-backend-config.md+ - Observer:+ - Data Extraction: pages/observer/data-extraction.md- API Reference:- API Guide: pages/api/index.md- n8n Node: https://pipelex.github.io/n8n-nodes-pipelex/@@ -168,6 +171,7 @@ nav:- Pipeline Tracker: pages/advanced-customization/pipeline-tracker-injection.md- Storage Provider: pages/advanced-customization/storage-provider-injection.md- Template Provider: pages/advanced-customization/template-provider-injection.md+ - Observer Provider: pages/advanced-customization/observer-provider-injection.md- Content Generator: pages/advanced-customization/content-generator-injection.md- Pipe Router: pages/advanced-customization/pipe-router-injection.md- License: license.mddiff --git a/pyproject.toml b/pyproject.tomlindex 6b0af50f6..87bcd38cc 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,7 +1,7 @@[project]name = "pipelex"-version = "0.13.2"-description = "Pipelex is an open-source dev tool based on a simple declarative language that lets you define replicable, structured, composable LLM pipelines."+version = "0.14.0"+description = "The open standard for repeatable AI workflows. Write business logic, not API calls."authors = [{ name = "Evotis S.A.S.", email = "[email protected]" }]maintainers = [{ name = "Pipelex staff", email = "[email protected]" }]license = "MIT"diff --git a/uv.lock b/uv.lockindex 2b19d373e..0fe0fbf61 100644--- a/uv.lock+++ b/uv.lock@@ -1900,7 +1900,7 @@ wheels = [[[package]]name = "pipelex"-version = "0.13.2"+version = "0.14.0"source = { editable = "." }dependencies = [{ name = "aiofiles" },