Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Change to PipelineInputs and add DictStuff in the PipelineInputs
thomashebrardb9241899f291
diff --git a/.cursor/rules/run_pipelex.mdc b/.cursor/rules/run_pipelex.mdcindex 5f29c45ab..eebaa223c 100644--- a/.cursor/rules/run_pipelex.mdc+++ b/.cursor/rules/run_pipelex.mdc@@ -81,7 +81,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.mdindex 2c65362a4..95f9c3b70 100644--- a/.github/copilot-instructions.md+++ b/.github/copilot-instructions.md@@ -927,7 +927,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/.windsurfrules.md b/.windsurfrules.mdindex 2c65362a4..95f9c3b70 100644--- a/.windsurfrules.md+++ b/.windsurfrules.md@@ -927,7 +927,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/AGENTS.md b/AGENTS.mdindex 2c65362a4..95f9c3b70 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -927,7 +927,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/BLACKBOX_RULES.md b/BLACKBOX_RULES.mdindex 2c65362a4..95f9c3b70 100644--- a/BLACKBOX_RULES.md+++ b/BLACKBOX_RULES.md@@ -927,7 +927,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex adb803736..7f77ba883 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -499,7 +499,7 @@ Simplified input memory:### Changed- Refactored `PipeInput` to use `InputRequirement` and `TypedNamedInputRequirement` classes instead of plain strings for input specifications.-- Updated `WorkingMemoryFactory` to handle `ImplicitMemory` instead of `CompactMemory`.+- Updated `WorkingMemoryFactory` to handle `PipelineInputs` instead of `CompactMemory`.- Replaced `ExecutePipelineException` with `PipelineInputError` in `execute_pipeline` function.- Updated `PipeBatch`, `PipeCondition`, `PipeParallel`, `PipeSequence`, `PipeFunc`, `PipeImgGen`, `PipeCompose`, `PipeLLM`, and `PipeExtract` classes to use `InputRequirement` for input handling.- Updated `PipeInput` creation in various test files to use `make_from_dict` method.diff --git a/CLAUDE.md b/CLAUDE.mdindex 2c65362a4..95f9c3b70 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -927,7 +927,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/docs/pages/api/index.md b/docs/pages/api/index.mdindex 0df19eb23..707ab51cd 100644--- a/docs/pages/api/index.md+++ b/docs/pages/api/index.md@@ -6,7 +6,7 @@ This guide covers everything you need to know about using the Pipelex API to exe1. [Quick Reference](#quick-reference)2. [Execute Pipeline](#execute-pipeline)-3. [Input Format: ImplicitMemory](#input-format-implicitmemory)+3. [Input Format: PipelineInputs](#input-format-implicitmemory)- [Case 1: Direct Content Format](#case-1-direct-content-format)- [Case 2: Explicit Format](#case-2-explicit-format-concept--content)4. [Search Domains Explained](#search-domains-explained)@@ -52,6 +52,9 @@ Authorization: Bearer YOUR_API_KEY// 5. Domain-prefixed concept (Case 2.5){ "data": { "concept": "accounting.Invoice", "content": { ... } } }++// 6. Full DictStuff with stuff_code (Case 2.7)+{ "invoice": { "stuff_code": "abc123", "stuff_name": "monthly_invoice", "concept": "Invoice", "content": { ... } } }```### Key Points@@ -91,7 +94,7 @@ Execute a Pipelex pipeline with flexible inputs.```**Request Fields:**-- `inputs` (ImplicitMemory): Flexible input format - see [Input Format](#input-format-implicitmemory) below+- `inputs` (PipelineInputs): Flexible input format - see [Input Format](#input-format-implicitmemory) below- `output_name` (string, optional): Name for the output slot- `output_multiplicity` (string, optional): Output multiplicity setting- `dynamic_output_concept_code` (string, optional): Override output concept@@ -121,13 +124,13 @@ Execute a Pipelex pipeline with flexible inputs.----## Input Format: ImplicitMemory+## Input Format: PipelineInputsRun your pipeline with flexible inputs that adapt to your needs. Pipelex supports multiple formats for providing inputs, making it easy to work with simple text, structured data, or complex objects.-### What is ImplicitMemory?+### What is PipelineInputs?-The `inputs` field uses **ImplicitMemory** format - a smart, flexible way to provide data to your pipelines. Instead of forcing you into a rigid structure, ImplicitMemory intelligently interprets your data based on how you provide it.+The `inputs` field uses **PipelineInputs** format - a smart, flexible way to provide data to your pipelines. Instead of forcing you into a rigid structure, PipelineInputs intelligently interprets your data based on how you provide it.### How Input Formatting Works@@ -147,6 +150,7 @@ The `inputs` field uses **ImplicitMemory** format - a smart, flexible way to pro- 2.4: List of StructuredContent objects with concept → `{"concept": "Invoice", "content": [...]}`- 2.5: Dictionary (structured data) → `{"concept": "Invoice", "content": {"field": "value"}}`- 2.6: List of dictionaries → `{"concept": "Invoice", "content": [{...}, {...}]}`+- 2.7: Full DictStuff (with stuff_code) → `{"stuff_code": "abc", "stuff_name": "name", "concept": "Invoice", "content": {...}}`**Pro Tip:** For **text inputs specifically**, skip the verbose format. Just provide the string directly: `"text": "Hello"` instead of `"text": {"concept": "Text", "content": "Hello"}`@@ -412,6 +416,62 @@ The system will:}```+### 2.7: DictStuff Format (Full Stuff Object)++The DictStuff format provides complete control over all Stuff fields, including `stuff_code` and `stuff_name`. This is useful when you need to preserve exact stuff identifiers from a previous pipeline execution or when working with serialized Stuff objects.++```json+{+ "inputs": {+ "invoice_data": {+ "stuff_code": "abc123",+ "stuff_name": "invoice_data",+ "concept": "accounting.Invoice",+ "content": {+ "invoice_number": "INV-001",+ "amount": 1250.00,+ "date": "2025-10-20"+ }+ }+ }+}+```++**Key Fields:**+- `stuff_code` (required): Unique identifier for the stuff object+- `stuff_name` (optional): Human-readable name for the stuff (Must be equal to the input key if provided)+- `concept` (required): Concept code (with optional domain prefix)+- `content` (required): The actual data content++**When to Use Case 2.7:**+- Passing serialized Stuff objects between pipeline executions+- When you need to preserve exact stuff identifiers+- Working with API responses that return full DictStuff objects+- Chaining pipelines where output from one becomes input to another++**Content Types:**+The `content` field in DictStuff can be:+- A dictionary (structured data)+- A string (for text-compatible concepts)+- A list (for list content)+- Any other format supported by the concept++**Important Validation Rules:**+- If both `stuff_name` in the DictStuff and the input parameter name are provided (not None), they **must match exactly**+- If only one is provided, that value will be used+- If neither is provided, the name will be generated from the concept+- Example of valid usage:+ ```json+ // ✅ Both names match+ { "stuff_name": "invoice", ... } with input key "invoice"++ // ✅ Only dict has name+ { "stuff_name": "my_invoice", ... } with no explicit input name++ // ❌ Names don't match - will raise error+ { "stuff_name": "invoice", ... } with input key "different_name"+ ```+---## Search Domains Explained@@ -577,7 +637,7 @@ print(response.pipe_output)### Combining PLX Content with Inputs-You can use all the ImplicitMemory input formats described above with `plx_content`:+You can use all the PipelineInputs input formats described above with `plx_content`:```json{diff --git a/docs/pages/build-reliable-ai-workflows-with-pipelex/executing-pipelines-with-inputs.md b/docs/pages/build-reliable-ai-workflows-with-pipelex/executing-pipelines-with-inputs.mdindex b15e9b17b..99eb77aed 100644--- a/docs/pages/build-reliable-ai-workflows-with-pipelex/executing-pipelines-with-inputs.md+++ b/docs/pages/build-reliable-ai-workflows-with-pipelex/executing-pipelines-with-inputs.md@@ -15,7 +15,7 @@ Under the hood, `input_memory` uses this type definition:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```Pipelex is smart about detecting what you mean and automatically wraps your data in the appropriate content types.diff --git a/pipelex/builder/builder_loop.py b/pipelex/builder/builder_loop.pyindex c38d5df5a..b57243ac0 100644--- a/pipelex/builder/builder_loop.py+++ b/pipelex/builder/builder_loop.py@@ -10,7 +10,7 @@ from pipelex.builder.builder_errors import (PipelexBundleUnexpectedError,)from pipelex.builder.builder_validation import validate_bundle_spec-from pipelex.client.protocol import ImplicitMemory+from pipelex.client.protocol import PipelineInputsfrom pipelex.core.pipes.pipe_blueprint import AllowedPipeCategoriesfrom pipelex.exceptions import StaticValidationErrorTypefrom pipelex.hub import get_required_pipe@@ -20,7 +20,7 @@ from pipelex.tools.misc.file_utils import save_text_to_pathclass BuilderLoop:- async def build_and_fix(self, pipe_code: str, input_memory: ImplicitMemory | None = None) -> PipelexBundleSpec:+ async def build_and_fix(self, pipe_code: str, input_memory: PipelineInputs | None = None) -> PipelexBundleSpec:pretty_print(f"Building and fixing with {pipe_code}")pipe_output = await execute_pipeline(pipe_code=pipe_code,diff --git a/pipelex/client/api_serializer.py b/pipelex/client/api_serializer.pyindex 4d5843d43..5519696f3 100644--- a/pipelex/client/api_serializer.py+++ b/pipelex/client/api_serializer.py@@ -4,7 +4,7 @@ from enum import Enumfrom pathlib import Pathfrom typing import Any, cast-from pipelex.client.protocol import ImplicitMemory+from pipelex.client.protocol import PipelineInputsfrom pipelex.core.memory.working_memory import WorkingMemory@@ -16,17 +16,17 @@ class ApiSerializer:FIELDS_TO_SKIP = ("__class__", "__module__")@classmethod- def serialize_working_memory_for_api(cls, working_memory: WorkingMemory | None = None) -> ImplicitMemory:+ def serialize_working_memory_for_api(cls, working_memory: WorkingMemory | None = None) -> PipelineInputs:"""Convert WorkingMemory to API-ready format using kajson with proper datetime handling.Args:working_memory: The WorkingMemory to serializeReturns:- ImplicitMemory ready for API transmission with datetime strings and no __class__/__module__.+ PipelineInputs ready for API transmission with datetime strings and no __class__/__module__."""- implicit_memory: ImplicitMemory = {}+ implicit_memory: PipelineInputs = {}if working_memory is None:return implicit_memorydiff --git a/pipelex/client/client.py b/pipelex/client/client.pyindex ab447487d..ff9495340 100644--- a/pipelex/client/client.py+++ b/pipelex/client/client.py@@ -5,7 +5,7 @@ from typing_extensions import overridefrom pipelex.client.pipeline_request_factory import PipelineRequestFactoryfrom pipelex.client.pipeline_response_factory import PipelineResponseFactory-from pipelex.client.protocol import ImplicitMemory, PipelexProtocol, PipelineResponse+from pipelex.client.protocol import PipelexProtocol, PipelineInputs, PipelineResponsefrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.exceptions import ClientAuthenticationError@@ -82,7 +82,7 @@ class PipelexClient(PipelexProtocol):self,pipe_code: str,working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,dynamic_output_concept_code: str | None = None,@@ -109,7 +109,7 @@ class PipelexClient(PipelexProtocol):self,pipe_code: str,working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,dynamic_output_concept_code: str | None = None,diff --git a/pipelex/client/pipeline_response_factory.py b/pipelex/client/pipeline_response_factory.pyindex cd58e4d90..fc0e32b9e 100644--- a/pipelex/client/pipeline_response_factory.py+++ b/pipelex/client/pipeline_response_factory.py@@ -1,8 +1,8 @@from typing import Any-from pipelex.client.protocol import PipelineResponse, PipelineState, SerializedPipeOutput, SerializedWorkingMemory-from pipelex.core.memory.working_memory import MAIN_STUFF_NAME, WorkingMemory-from pipelex.core.pipes.pipe_output import PipeOutput+from pipelex.client.protocol import PipelineResponse, PipelineState+from pipelex.core.memory.working_memory import MAIN_STUFF_NAME, DictWorkingMemory, WorkingMemory+from pipelex.core.pipes.pipe_output import DictPipeOutput, PipeOutputfrom pipelex.core.stuffs.stuff import DictStuff@@ -10,7 +10,7 @@ class PipelineResponseFactory:"""Factory class for creating PipelineResponse objects from PipeOutput."""@staticmethod- def _serialize_working_memory_with_dict_stuffs(working_memory: WorkingMemory) -> SerializedWorkingMemory:+ def _serialize_working_memory_with_dict_stuffs(working_memory: WorkingMemory) -> DictWorkingMemory:"""Convert WorkingMemory to dict with DictStuff objects (content as dict).Keeps the WorkingMemory structure but converts each Stuff.content to dict.@@ -28,12 +28,12 @@ class PipelineResponseFactory:dict_stuff = DictStuff(stuff_code=stuff.stuff_code,stuff_name=stuff.stuff_name,- concept=stuff.concept,+ concept=stuff.concept.concept_string,content=stuff.content.model_dump(serialize_as_any=True),)dict_stuffs_root[stuff_name] = dict_stuff- return SerializedWorkingMemory(root=dict_stuffs_root, aliases=working_memory.aliases)+ return DictWorkingMemory(root=dict_stuffs_root, aliases=working_memory.aliases)@staticmethoddef make_from_pipe_output(@@ -68,7 +68,7 @@ class PipelineResponseFactory:created_at=created_at,pipeline_state=pipeline_state,finished_at=finished_at,- pipe_output=SerializedPipeOutput(+ pipe_output=DictPipeOutput(working_memory=PipelineResponseFactory._serialize_working_memory_with_dict_stuffs(pipe_output.working_memory),pipeline_run_id=pipe_output.pipeline_run_id,),diff --git a/pipelex/client/protocol.py b/pipelex/client/protocol.pyindex bacc3c0f9..2249129d6 100644--- a/pipelex/client/protocol.py+++ b/pipelex/client/protocol.py@@ -6,6 +6,7 @@ from pydantic.functional_validators import SkipValidationfrom typing_extensions import Annotated, runtime_checkablefrom pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.pipes.pipe_output import DictPipeOutputfrom pipelex.core.stuffs.stuff import DictStufffrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.pipe_run.pipe_run_params import PipeOutputMultiplicity@@ -25,26 +26,17 @@ from pipelex.types import StrEnum# - 2.4: {"concept": str, "content": Sequence[StuffContent]}# - 2.5: {"concept": str, "content": dict[str, Any]}# - 2.6: {"concept": str, "content": Sequence[dict[str, Any]]}-DictStuffContent = dict[str, Any]+# - 2.7: DictStuff {"stuff_code": str, "stuff_name": str | None, "concept": str, "content": Any}+ReducedDictStuff = dict[str, Any] # Only contains {"concept": str, "content": Any}StuffContentOrData = (str # Case 1.1| Sequence[str] # Case 1.2| StuffContent # Case 1.3 (also covers Case 1.5 as ListContent is a StuffContent)| Sequence[StuffContent] # Case 1.4 (covariant - accepts list[TextContent], etc.)- | DictStuffContent # Case 2.1, 2.2, 2.3, 2.4, 2.5, 2.6+ | ReducedDictStuff # Case 2.1, 2.2, 2.3, 2.4, 2.5, 2.6+ | DictStuff # Case 2.7)-DictMemory = dict[str, DictStuffContent] # Serialized dict format (no StuffContent objects)-ImplicitMemory = dict[str, StuffContentOrData] # Can include both dict and StuffContent---class SerializedWorkingMemory(BaseModel):- root: dict[str, DictStuff]- aliases: dict[str, str]---class SerializedPipeOutput(BaseModel):- working_memory: SerializedWorkingMemory- pipeline_run_id: str+PipelineInputs = dict[str, StuffContentOrData] # Can include both dict and StuffContentclass PipelineState(StrEnum):@@ -77,7 +69,7 @@ class PipelineRequest(BaseModel):"""Request for executing a pipeline.Attributes:- inputs (ImplicitMemory | None): Inputs in ImplicitMemory format - Pydantic validation is skipped+ inputs (PipelineInputs | None): Inputs in PipelineInputs format - Pydantic validation is skippedto preserve the flexible format (dicts, strings, StuffContent objects, etc.)output_name (str | None): Name of the output slot to write tooutput_multiplicity (PipeOutputMultiplicity | None): Output multiplicity setting@@ -86,7 +78,7 @@ class PipelineRequest(BaseModel):"""- inputs: Annotated[ImplicitMemory | None, SkipValidation] = None+ inputs: Annotated[PipelineInputs | WorkingMemory | None, SkipValidation] = Noneoutput_name: str | None = Noneoutput_multiplicity: PipeOutputMultiplicity | None = Nonedynamic_output_concept_code: str | None = None@@ -101,7 +93,7 @@ class PipelineResponse(ApiResponse):created_at (str): Timestamp when the pipeline was createdpipeline_state (PipelineState): Current state of the pipelinefinished_at (str | None): Timestamp when the pipeline finished, if completed- pipe_output (SerializedPipeOutput | None): Output data from the pipeline execution (working_memory dict + pipeline_run_id)+ pipe_output (DictPipeOutput | None): Output data from the pipeline execution (working_memory dict + pipeline_run_id)main_stuff_name (str | None): Name of the main stuff in the pipeline outputpipe_structures (dict[str, Any] | None): Structure of the pipeline to execute@@ -111,7 +103,7 @@ class PipelineResponse(ApiResponse):created_at: strpipeline_state: PipelineStatefinished_at: str | None = None- pipe_output: SerializedPipeOutput | None = None+ pipe_output: DictPipeOutput | None = Nonemain_stuff_name: str | None = Nonepipe_structures: dict[str, Any] | None = None@@ -137,7 +129,7 @@ class PipelexProtocol(Protocol):self,pipe_code: str,working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,dynamic_output_concept_code: str | None = None,@@ -147,7 +139,7 @@ class PipelexProtocol(Protocol):Args:pipe_code (str): The code identifying the pipeline to executeworking_memory (WorkingMemory | None): Memory context passed to the pipeline- inputs (ImplicitMemory | None): Inputs passed to the pipeline+ inputs (PipelineInputs | None): Inputs passed to the pipelineoutput_name (str | None): Target output slot nameoutput_multiplicity (PipeOutputMultiplicity | None): Output multiplicity settingdynamic_output_concept_code (str | None): Override for dynamic output concept@@ -166,7 +158,7 @@ class PipelexProtocol(Protocol):self,pipe_code: str,working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,dynamic_output_concept_code: str | None = None,@@ -176,7 +168,7 @@ class PipelexProtocol(Protocol):Args:pipe_code (str): The code identifying the pipeline to executeworking_memory (WorkingMemory | None): Memory context passed to the pipeline- inputs (ImplicitMemory | None): Inputs passed to the pipeline+ inputs (PipelineInputs | None): Inputs passed to the pipelineoutput_name (str | None): Target output slot nameoutput_multiplicity (PipeOutputMultiplicity | None): Output multiplicity settingdynamic_output_concept_code (str | None): Override for dynamic output conceptdiff --git a/pipelex/core/concepts/concept.py b/pipelex/core/concepts/concept.pyindex ed99e6f6a..3b9889e0b 100644--- a/pipelex/core/concepts/concept.py+++ b/pipelex/core/concepts/concept.py@@ -18,7 +18,7 @@ from pipelex.types import StrEnumclass Concept(BaseModel):- model_config = ConfigDict(extra="ignore", strict=True)+ model_config = ConfigDict(extra="forbid", strict=True)code: strdomain: strdiff --git a/pipelex/core/memory/working_memory.py b/pipelex/core/memory/working_memory.pyindex 75cd20819..ad9294611 100644--- a/pipelex/core/memory/working_memory.py+++ b/pipelex/core/memory/working_memory.py@@ -11,7 +11,7 @@ from pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.mermaid_content import MermaidContentfrom pipelex.core.stuffs.number_content import NumberContentfrom pipelex.core.stuffs.pdf_content import PDFContent-from pipelex.core.stuffs.stuff import Stuff+from pipelex.core.stuffs.stuff import DictStuff, Stufffrom pipelex.core.stuffs.stuff_artefact import StuffArtefactfrom pipelex.core.stuffs.stuff_content import StuffContentTypefrom pipelex.core.stuffs.text_and_images_content import TextAndImagesContent@@ -34,6 +34,11 @@ StuffDict = dict[str, Stuff]StuffArtefactDict = dict[str, StuffArtefact]+class DictWorkingMemory(BaseModel):+ root: dict[str, DictStuff]+ aliases: dict[str, str]++class WorkingMemory(BaseModel, ContextProviderAbstract):root: StuffDict = Field(default_factory=dict)aliases: dict[str, str] = Field(default_factory=dict)diff --git a/pipelex/core/memory/working_memory_factory.py b/pipelex/core/memory/working_memory_factory.pyindex 40e092f61..a16bd0ba9 100644--- a/pipelex/core/memory/working_memory_factory.py+++ b/pipelex/core/memory/working_memory_factory.py@@ -5,7 +5,7 @@ from polyfactory.factories.pydantic_factory import ModelFactoryfrom pydantic import BaseModelfrom pipelex import log-from pipelex.client.protocol import ImplicitMemory+from pipelex.client.protocol import PipelineInputsfrom pipelex.core.concepts.concept import ConceptBlueprint, SpecialDomainfrom pipelex.core.concepts.concept_native import NativeConceptCodefrom pipelex.core.memory.working_memory import MAIN_STUFF_NAME, StuffDict, WorkingMemory@@ -125,7 +125,7 @@ class WorkingMemoryFactory(BaseModel):@classmethoddef make_from_implicit_memory(cls,- implicit_memory: ImplicitMemory,+ implicit_memory: PipelineInputs,search_domains: list[str] | None = None,) -> WorkingMemory:"""Create a WorkingMemory from a implicit memory dictionary.diff --git a/pipelex/core/pipes/pipe_output.py b/pipelex/core/pipes/pipe_output.pyindex c20667efe..9d1960964 100644--- a/pipelex/core/pipes/pipe_output.py+++ b/pipelex/core/pipes/pipe_output.py@@ -2,7 +2,7 @@ from typing import TypeVarfrom pydantic import BaseModel, Field-from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.memory.working_memory import DictWorkingMemory, WorkingMemoryfrom pipelex.core.stuffs.html_content import HtmlContentfrom pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.list_content import ListContent@@ -15,6 +15,11 @@ from pipelex.core.stuffs.text_content import TextContentfrom pipelex.pipeline.pipeline_models import SpecialPipelineId+class DictPipeOutput(BaseModel):+ working_memory: DictWorkingMemory+ pipeline_run_id: str++class PipeOutput(BaseModel):working_memory: WorkingMemory = Field(default_factory=WorkingMemory)pipeline_run_id: str = Field(default=SpecialPipelineId.UNTITLED)diff --git a/pipelex/core/stuffs/stuff.py b/pipelex/core/stuffs/stuff.pyindex 4740c20fe..03dfa2622 100644--- a/pipelex/core/stuffs/stuff.py+++ b/pipelex/core/stuffs/stuff.py@@ -21,7 +21,7 @@ from pipelex.tools.typing.pydantic_utils import CustomBaseModel, format_pydanticclass Stuff(CustomBaseModel):- model_config = ConfigDict(extra="ignore", strict=True)+ model_config = ConfigDict(extra="forbid", strict=True)stuff_code: strstuff_name: str | None = None@@ -200,9 +200,9 @@ class DictStuff(CustomBaseModel):Has the exact same structure as Stuff but with dict content."""- model_config = ConfigDict(extra="ignore", strict=True)+ model_config = ConfigDict(extra="forbid", strict=True)stuff_code: strstuff_name: str | None = None- concept: Concept+ concept: strcontent: dict[str, Any]diff --git a/pipelex/core/stuffs/stuff_factory.py b/pipelex/core/stuffs/stuff_factory.pyindex 81f32640f..c40ab416a 100644--- a/pipelex/core/stuffs/stuff_factory.py+++ b/pipelex/core/stuffs/stuff_factory.py@@ -148,6 +148,8 @@ class StuffFactory:2.4: {"concept": "...", "content": list[StuffContent]} → ListContent[StuffContent]2.5: {"concept": "...", "content": dict} → Create StuffContent from dict2.6: {"concept": "...", "content": list[dict]} → ListContent[StuffContent] from dicts+ 2.7: DictStuff (full stuff dict with stuff_code, stuff_name, concept, content) → Create Stuff from all fields+ Note: If both stuff_name in dict and name argument are provided, they must match exactly"""concept_library = get_concept_library()@@ -326,6 +328,70 @@ class StuffFactory:msg = f"Trying to create a Stuff '{name}' from a dict that should represent a StuffContentOrData but does not have a 'content' key."raise StuffFactoryError(msg)+ # Case 2.7: DictStuff - full stuff dict with stuff_code, stuff_name, concept, content+ if "stuff_code" in stuff_content_or_data:+ # This is a DictStuff - it has all the fields of a Stuff+ if len(stuff_content_or_data) not in (3, 4): # 3 if no stuff_name, 4 if stuff_name present+ msg = (+ f"Trying to create a Stuff '{name}' from a DictStuff but it does not have the correct keys. "+ "Expected keys: 'stuff_code', 'concept', 'content', and optionally 'stuff_name'."+ )+ raise StuffFactoryError(msg)++ dict_stuff_code = stuff_content_or_data["stuff_code"]+ dict_stuff_name_in_dict = stuff_content_or_data.get("stuff_name")+ concept_string = stuff_content_or_data["concept"]+ content = stuff_content_or_data["content"]++ # Validate: if both dict_stuff_name and name argument are provided, they must match+ if dict_stuff_name_in_dict is not None and name is not None and dict_stuff_name_in_dict != name:+ msg = (+ f"Trying to create a Stuff from a DictStuff but the stuff_name in the dict ('{dict_stuff_name_in_dict}') "+ f"does not match the name argument ('{name}'). They must be equal if both are provided."+ )+ raise StuffFactoryError(msg)++ # Use dict's stuff_name if present, otherwise fallback to name argument+ dict_stuff_name = dict_stuff_name_in_dict if dict_stuff_name_in_dict is not None else name++ # Get the concept from the library+ try:+ concept = concept_library.get_required_concept_from_concept_string_or_code(+ concept_string_or_code=concept_string, search_domains=search_domains+ )+ except ConceptLibraryConceptNotFoundError as exc:+ msg = f"Trying to create a Stuff from a DictStuff but the concept of name '{concept_string}' is not found in the library"+ raise StuffFactoryError(msg) from exc++ # Handle content based on type (similar to Case 2.5 for dict content)+ if isinstance(content, dict):+ the_class = get_class_registry().get_class(name=concept.structure_class_name)+ if the_class is None:+ msg = (+ f"Trying to create a Stuff from a DictStuff but the concept of name '{concept_string}' is not compatible with a dict content"+ )+ raise StuffFactoryError(msg)++ return cls.make_stuff(+ name=dict_stuff_name,+ code=dict_stuff_code,+ concept=concept,+ content=the_class.model_validate(obj=content),+ )+ else:+ # Content might be other types (str, list, etc.) - handle recursively+ stuff_content = StuffContentFactory.make_stuff_content_from_concept_with_fallback(+ concept=concept,+ value=content,+ )+ return cls.make_stuff(+ name=dict_stuff_name,+ code=dict_stuff_code,+ concept=concept,+ content=stuff_content,+ )++ # ReducedDictStuff - only concept and contentif len(stuff_content_or_data) != 2:msg = (f"Trying to create a Stuff '{name}' from a dict that should represent a StuffContentOrData but does not have "diff --git a/pipelex/kit/agent_rules/run_pipelex.md b/pipelex/kit/agent_rules/run_pipelex.mdindex 21787573f..c9241dd75 100644--- a/pipelex/kit/agent_rules/run_pipelex.md+++ b/pipelex/kit/agent_rules/run_pipelex.md@@ -75,7 +75,7 @@ pretty_print(gantt_chart, title="Gantt Chart")The input memory is a dictionary, where the key is the name of the input variable and the value provides details to make it a stuff object. The relevant definitions are:```pythonStuffContentOrData = dict[str, Any] | StuffContent | list[Any] | str-ImplicitMemory = dict[str, StuffContentOrData]+PipelineInputs = dict[str, StuffContentOrData]```As you can seen, we made it so different ways can be used to define that stuff using structured content or data.diff --git a/pipelex/pipeline/execute.py b/pipelex/pipeline/execute.pyindex 8e63e6293..70ae7af99 100644--- a/pipelex/pipeline/execute.py+++ b/pipelex/pipeline/execute.py@@ -1,8 +1,7 @@-from pipelex.client.protocol import ImplicitMemory+from pipelex.client.protocol import PipelineInputsfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.pipes.pipe_output import PipeOutput-from pipelex.exceptions import PipelineInputErrorfrom pipelex.hub import (get_pipe_router,get_pipeline_manager,@@ -22,8 +21,7 @@ from pipelex.system.environment import get_optional_envasync def execute_pipeline(pipe_code: str,- working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | WorkingMemory | None = None,search_domains: list[str] | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,@@ -40,10 +38,8 @@ async def execute_pipeline(----------pipe_code:The code of the pipe to execute.- working_memory:- Optional ``WorkingMemory`` instance passed to the pipe.inputs:- Optional implicit memory to pass to the pipe.+ Optional implicit memory or working memory to pass to the pipe.output_name:Name of the output slot to write to.output_multiplicity:@@ -69,15 +65,16 @@ async def execute_pipeline(if pipe.domain not in search_domains:search_domains.insert(0, pipe.domain)- # Can be either working_memory or implicit_memory or neither, but not both- if working_memory and inputs:- msg = f"Cannot pass both working_memory and inputs to `execute_pipeline` {pipe_code=}"- raise PipelineInputError(msg)+ working_memory: WorkingMemory | None = None+if inputs:- working_memory = WorkingMemoryFactory.make_from_implicit_memory(- implicit_memory=inputs,- search_domains=search_domains,- )+ if isinstance(inputs, WorkingMemory):+ working_memory = inputs+ else:+ working_memory = WorkingMemoryFactory.make_from_implicit_memory(+ implicit_memory=inputs,+ search_domains=search_domains,+ )if pipe_run_mode is None:if run_mode_from_env := get_optional_env(key=FORCE_DRY_RUN_MODE_ENV_KEY):diff --git a/pipelex/pipeline/start.py b/pipelex/pipeline/start.pyindex 27fa0d1fa..f5c4e58af 100644--- a/pipelex/pipeline/start.py+++ b/pipelex/pipeline/start.py@@ -1,6 +1,6 @@import asyncio-from pipelex.client.protocol import ImplicitMemory+from pipelex.client.protocol import PipelineInputsfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.pipes.pipe_output import PipeOutput@@ -15,7 +15,7 @@ from pipelex.pipeline.job_metadata import JobMetadataasync def start_pipeline(pipe_code: str,working_memory: WorkingMemory | None = None,- inputs: ImplicitMemory | None = None,+ inputs: PipelineInputs | None = None,output_name: str | None = None,output_multiplicity: PipeOutputMultiplicity | None = None,dynamic_output_concept_code: str | None = None,diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.pyindex 125d8a7e4..579f92ab6 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py@@ -65,7 +65,7 @@ async def test_review_analysis_sequence_with_batching(pipe_run_mode: PipeRunMode# Execute the pipelinepipe_output = await execute_pipeline(pipe_code="analyze_reviews_sequence",- working_memory=working_memory,+ inputs=working_memory,)# Basic output validationdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.pyindex 3a3ca2caf..fc17917c1 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py@@ -34,7 +34,7 @@ class TestPipeSequenceMultiplicity:# Execute the pipelinepipe_output = await execute_pipeline(pipe_code="creative_ideation_sequence",- working_memory=working_memory,+ inputs=working_memory,pipe_run_mode=pipe_run_mode,)diff --git a/tests/integration/pipelex/client/test_client.py b/tests/integration/pipelex/client/test_client.pyindex 7f4c08638..4340ac2ea 100644--- a/tests/integration/pipelex/client/test_client.py+++ b/tests/integration/pipelex/client/test_client.py@@ -124,7 +124,7 @@ class TestPipelexApiClient:# Verify main_stuff structureassert working_memory.root["main_stuff"] is not None- assert working_memory.root["main_stuff"].concept.code == "RetrievedExcerpt"+ assert working_memory.root["main_stuff"].concept == "RetrievedExcerpt"assert working_memory.root["main_stuff"].content is not Noneassert len(working_memory.root["main_stuff"].content) > 0diff --git a/tests/integration/pipelex/core/memory/test_working_memory_factory.py b/tests/integration/pipelex/core/memory/test_working_memory_factory.pyindex da010971c..08b547e4c 100644--- a/tests/integration/pipelex/core/memory/test_working_memory_factory.py+++ b/tests/integration/pipelex/core/memory/test_working_memory_factory.py@@ -8,12 +8,12 @@ from pipelex.core.stuffs.text_and_images_content import TextAndImagesContentfrom pipelex.core.stuffs.text_content import TextContentif TYPE_CHECKING:- from pipelex.client.protocol import ImplicitMemory+ from pipelex.client.protocol import PipelineInputsclass TestWorkingMemoryFactory:def test_make_from_compact_memory_with_text_content(self):- implicit_memory: ImplicitMemory = {+ implicit_memory: PipelineInputs = {"text_item": {"concept": NativeConceptCode.TEXT,"content": "Hello, world!",@@ -32,7 +32,7 @@ class TestWorkingMemoryFactory:def test_make_from_compact_memory_with_complex_nested_content(self):"""Test deserialization of compact memory with complex nested structured content."""- implicit_memory: ImplicitMemory = {+ implicit_memory: PipelineInputs = {"complex_page": {"concept": NativeConceptCode.PAGE,"content": {@@ -113,7 +113,7 @@ class TestWorkingMemoryFactory:def test_make_from_compact_memory_empty(self):"""Test deserialization of empty compact memory."""- implicit_memory: ImplicitMemory = {}+ implicit_memory: PipelineInputs = {}working_memory = WorkingMemoryFactory.make_from_implicit_memory(implicit_memory=implicit_memory)@@ -122,7 +122,7 @@ class TestWorkingMemoryFactory:def test_make_from_compact_memory_multiple_items(self):"""Test deserialization of compact memory with multiple items."""- implicit_memory: ImplicitMemory = {+ implicit_memory: PipelineInputs = {"text1": {"concept": NativeConceptCode.TEXT,"content": "First text",diff --git a/tests/unit/core/stuffs/data.py b/tests/unit/core/stuffs/data.pyindex 390ad0880..aa7a6da22 100644--- a/tests/unit/core/stuffs/data.py+++ b/tests/unit/core/stuffs/data.py@@ -26,7 +26,7 @@ class AnotherConcept(StructuredContent):value: int = Field(description="Value field")-TEST_CASES: list[tuple[str, StuffContentOrData, str, str, Stuff]] = [+TEST_CASES: list[tuple[str, StuffContentOrData, str | None, str, Stuff]] = [# Case 1.1: Content is a string("case-1.1-string",@@ -367,11 +367,92 @@ TEST_CASES: list[tuple[str, StuffContentOrData, str, str, Stuff]] = [),),),+ # Case 2.7: DictStuff with all fields (stuff_code, stuff_name, concept, content)+ # Note: name argument must match stuff_name in dict if both are provided+ (+ "case-2.7-dict-stuff-with-stuff-name",+ {+ "stuff_code": "custom_code",+ "stuff_name": "custom_name",+ "concept": "test_domain.MyConcept",+ "content": {+ "arg1": "something",+ "arg2": 1,+ "arg3": {"arg4": "something else"},+ },+ },+ "custom_name", # Must match the stuff_name in the dict+ "ignored_code",+ Stuff(+ stuff_code="custom_code",+ stuff_name="custom_name",+ concept=ConceptFactory.make(+ concept_code="MyConcept",+ domain="test_domain",+ description="Test concept for unit tests",+ structure_class_name="MyConcept",+ ),+ content=MyConcept(arg1="something", arg2=1, arg3=MySubClass(arg4="something else")),+ ),+ ),+ # Case 2.7a: DictStuff with stuff_name, but name argument is None+ (+ "case-2.7a-dict-stuff-with-stuff-name-no-arg",+ {+ "stuff_code": "custom_code_a",+ "stuff_name": "custom_name_a",+ "concept": "test_domain.MyConcept",+ "content": {+ "arg1": "something",+ "arg2": 1,+ "arg3": {"arg4": "something else"},+ },+ },+ None, # name argument is None, so dict's stuff_name is used+ "ignored_code",+ Stuff(+ stuff_code="custom_code_a",+ stuff_name="custom_name_a",+ concept=ConceptFactory.make(+ concept_code="MyConcept",+ domain="test_domain",+ description="Test concept for unit tests",+ structure_class_name="MyConcept",+ ),+ content=MyConcept(arg1="something", arg2=1, arg3=MySubClass(arg4="something else")),+ ),+ ),+ # Case 2.7b: DictStuff without stuff_name (only 3 keys)+ (+ "case-2.7b-dict-stuff-without-stuff-name",+ {+ "stuff_code": "custom_code_2",+ "concept": "test_domain.MyConcept",+ "content": {+ "arg1": "arg1_value",+ "arg2": 42,+ "arg3": {"arg4": "arg4_value"},+ },+ },+ "fallback_name",+ "ignored_code",+ Stuff(+ stuff_code="custom_code_2",+ stuff_name="fallback_name",+ concept=ConceptFactory.make(+ concept_code="MyConcept",+ domain="test_domain",+ description="Test concept for unit tests",+ structure_class_name="MyConcept",+ ),+ content=MyConcept(arg1="arg1_value", arg2=42, arg3=MySubClass(arg4="arg4_value")),+ ),+ ),]# Test cases with search_domains - format: (test_name, stuff_content_or_data, stuff_name, stuff_code, search_domains, expected_stuff)-SEARCH_DOMAIN_TEST_CASES: list[tuple[str, StuffContentOrData, str, str, list[str], Stuff]] = [+SEARCH_DOMAIN_TEST_CASES: list[tuple[str, StuffContentOrData, str | None, str, list[str], Stuff]] = [# Case with search_domains: StuffContent object("search-domain-stuff-content",@@ -473,7 +554,7 @@ SEARCH_DOMAIN_TEST_CASES: list[tuple[str, StuffContentOrData, str, str, list[str# Error test cases - these should raise exceptions-ERROR_TEST_CASES: list[tuple[str, StuffContentOrData, str, str, list[str] | None, type[Exception], str]] = [+ERROR_TEST_CASES: list[tuple[str, StuffContentOrData, str | None, str, list[str] | None, type[Exception], str]] = [# Format: (test_name, stuff_content_or_data, stuff_name, stuff_code, search_domains, expected_exception, error_match)# Empty list - should fail(@@ -575,4 +656,67 @@ ERROR_TEST_CASES: list[tuple[str, StuffContentOrData, str, str, list[str] | NoneException,"not compatible",),+ # DictStuff with wrong number of keys - should fail+ (+ "error-dict-stuff-wrong-keys",+ {+ "stuff_code": "code",+ "concept": "Text",+ "content": "text",+ "extra": "key",+ "another": "key",+ },+ "stuff_name",+ "stuff_code",+ None,+ Exception,+ "correct keys",+ ),+ # DictStuff with missing concept - should fail+ (+ "error-dict-stuff-missing-concept",+ {+ "stuff_code": "code",+ "stuff_name": "name",+ "content": {"field": "value"},+ },+ "stuff_name",+ "stuff_code",+ None,+ Exception,+ "'concept' key",+ ),+ # DictStuff with concept not found - should fail+ (+ "error-dict-stuff-concept-not-found",+ {+ "stuff_code": "code",+ "concept": "NonExistentConcept",+ "content": {"field": "value"},+ },+ "stuff_name",+ "stuff_code",+ None,+ Exception,+ "not found",+ ),+ # DictStuff with mismatched stuff_name - should fail+ (+ "error-dict-stuff-name-mismatch",+ {+ "stuff_code": "code",+ "stuff_name": "name_in_dict",+ "concept": "test_domain.MyConcept",+ "content": {+ "arg1": "arg1",+ "arg2": 1,+ "arg3": {"arg4": "arg4"},+ },+ },+ "different_name", # Does not match "name_in_dict"+ "stuff_code",+ ["test_domain"],+ Exception,+ "does not match",+ ),]diff --git a/tests/unit/core/stuffs/test_stuff_factory_implicit_memory.py b/tests/unit/core/stuffs/test_stuff_factory_implicit_memory.pyindex 7831eba7f..c13050b01 100644--- a/tests/unit/core/stuffs/test_stuff_factory_implicit_memory.py+++ b/tests/unit/core/stuffs/test_stuff_factory_implicit_memory.py@@ -82,6 +82,7 @@ class TestStuffFactoryImplicitMemory:- List of StuffContent objects (2.4)- Dict content (2.5)- List of dicts (2.6)+ - DictStuff with all fields (2.7)"""@pytest.mark.parametrize(@@ -93,7 +94,7 @@ class TestStuffFactoryImplicitMemory:setup_test_concept: Any,test_name: str,stuff_content_or_data: StuffContentOrData,- stuff_name: str,+ stuff_name: str | None,stuff_code: str,expected_stuff: Stuff,):@@ -126,7 +127,7 @@ class TestStuffFactoryImplicitMemoryWithSearchDomains:setup_test_concept: Any,test_name: str,stuff_content_or_data: StuffContentOrData,- stuff_name: str,+ stuff_name: str | None,stuff_code: str,search_domains: list[str],expected_stuff: Stuff,@@ -161,7 +162,7 @@ class TestStuffFactoryImplicitMemoryErrors:setup_test_concept: Any,test_name: str,stuff_content_or_data: StuffContentOrData,- stuff_name: str,+ stuff_name: str | None,stuff_code: str,search_domains: list[str] | None,expected_exception: type[Exception],