Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Docs/rules to call cli (#399)
Louis Choquel8d745feec1b9
diff --git a/.cursor/rules/run_pipelex.mdc b/.cursor/rules/run_pipelex.mdcindex 838783897..78aed7449 100644--- a/.cursor/rules/run_pipelex.mdc+++ b/.cursor/rules/run_pipelex.mdc@@ -6,6 +6,21 @@ globs:---# Guide to execute a pipeline and write example code+## Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+# Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+# or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+## Example to execute a pipeline with text output```pythondiff --git a/.cursor/rules/write_pipelex.mdc b/.cursor/rules/write_pipelex.mdcindex 841da9765..2d977d3af 100644--- a/.cursor/rules/write_pipelex.mdc+++ b/.cursor/rules/write_pipelex.mdc@@ -8,7 +8,10 @@ globs:# Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)## Pipeline File Naming@@ -850,6 +853,10 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.mdindex 06a223540..9f0a61293 100644--- a/.github/copilot-instructions.md+++ b/.github/copilot-instructions.md@@ -1,8 +1,13 @@<!-- BEGIN_PIPELEX_RULES -->+# Pipelex Rules+## Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)### Pipeline File Naming@@ -844,12 +849,31 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.## Guide to execute a pipeline and write example code+### Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+## Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+## or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+### Example to execute a pipeline with text output```pythondiff --git a/.windsurfrules.md b/.windsurfrules.mdindex 06a223540..9f0a61293 100644--- a/.windsurfrules.md+++ b/.windsurfrules.md@@ -1,8 +1,13 @@<!-- BEGIN_PIPELEX_RULES -->+# Pipelex Rules+## Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)### Pipeline File Naming@@ -844,12 +849,31 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.## Guide to execute a pipeline and write example code+### Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+## Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+## or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+### Example to execute a pipeline with text output```pythondiff --git a/AGENTS.md b/AGENTS.mdindex 06a223540..9f0a61293 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -1,8 +1,13 @@<!-- BEGIN_PIPELEX_RULES -->+# Pipelex Rules+## Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)### Pipeline File Naming@@ -844,12 +849,31 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.## Guide to execute a pipeline and write example code+### Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+## Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+## or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+### Example to execute a pipeline with text output```pythondiff --git a/BLACKBOX_RULES.md b/BLACKBOX_RULES.mdindex 06a223540..9f0a61293 100644--- a/BLACKBOX_RULES.md+++ b/BLACKBOX_RULES.md@@ -1,8 +1,13 @@<!-- BEGIN_PIPELEX_RULES -->+# Pipelex Rules+## Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)### Pipeline File Naming@@ -844,12 +849,31 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.## Guide to execute a pipeline and write example code+### Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+## Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+## or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+### Example to execute a pipeline with text output```pythondiff --git a/CLAUDE.md b/CLAUDE.mdindex 06a223540..9f0a61293 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -1,8 +1,13 @@<!-- BEGIN_PIPELEX_RULES -->+# Pipelex Rules+## Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)### Pipeline File Naming@@ -844,12 +849,31 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.## Guide to execute a pipeline and write example code+### Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+## Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+## or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+### Example to execute a pipeline with text output```pythondiff --git a/pipelex/core/memory/working_memory.py b/pipelex/core/memory/working_memory.pyindex 897ce02a0..5889785b2 100644--- a/pipelex/core/memory/working_memory.py+++ b/pipelex/core/memory/working_memory.py@@ -78,6 +78,9 @@ class WorkingMemory(BaseModel, ContextProviderAbstract):def get_optional_main_stuff(self) -> Stuff | None:return self.get_optional_stuff(name=MAIN_STUFF_NAME)+ def get_main_stuff(self) -> Stuff:+ return self.get_stuff(name=MAIN_STUFF_NAME)+# TODO: all calls to get_stuff should catch WorkingMemoryStuffNotFoundError in order to indicate what pipe is missing a required stuffdef get_stuff(self, name: str) -> Stuff:if named_stuff := self.root.get(name):@@ -111,9 +114,6 @@ class WorkingMemory(BaseModel, ContextProviderAbstract):def is_stuff_code_used(self, stuff_code: str) -> bool:return any(stuff.concept.code == stuff_code for stuff in self.root.values())- def get_main_stuff(self) -> Stuff:- return self.get_stuff(name=MAIN_STUFF_NAME)-def remove_stuff(self, name: str):self.root.pop(name, None)diff --git a/pipelex/kit/agent_rules/run_pipelex.md b/pipelex/kit/agent_rules/run_pipelex.mdindex 2f146d388..42d696587 100644--- a/pipelex/kit/agent_rules/run_pipelex.md+++ b/pipelex/kit/agent_rules/run_pipelex.md@@ -1,5 +1,20 @@# Guide to execute a pipeline and write example code+## Prerequisites: Virtual Environment++**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work.++For standard installations, the virtual environment is named `.venv`. Always check this first:++```bash+# Activate the virtual environment (standard installation)+source .venv/bin/activate # On macOS/Linux+# or+.venv\Scripts\activate # On Windows+```++If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active.+## Example to execute a pipeline with text output```pythondiff --git a/pipelex/kit/agent_rules/write_pipelex.md b/pipelex/kit/agent_rules/write_pipelex.mdindex edbda188f..d6e0cc109 100644--- a/pipelex/kit/agent_rules/write_pipelex.md+++ b/pipelex/kit/agent_rules/write_pipelex.md@@ -1,7 +1,10 @@# Guide to write or edit pipelines using the Pipelex language in .plx files- Always first write your "plan" in natural language, then transcribe it in pipelex.-- You should ALWAYS RUN the terminal command `make validate` when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+- You should ALWAYS RUN validation when you are writing or editing a `.plx` file. It will ensure the pipe is runnable. If not, iterate.+ - For a specific file: `pipelex validate path_to_file.plx`+ - For all pipelines: `pipelex validate all`+ - **IMPORTANT**: Ensure the Python virtual environment is activated before running `pipelex` commands. For standard installations, the venv is named `.venv` - always check that first. The commands will not work without proper venv activation.- Please use POSIX standard for files. (empty lines, no trailing whitespaces, etc.)## Pipeline File Naming@@ -843,6 +846,10 @@ You can override the predefined llm presets by setting them in `.pipelex/inferen----ALWAYS RUN `make validate` when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+ALWAYS RUN validation when you are finished writing pipelines: This checks for errors. If there are errors, iterate until it works.+- For a specific bundle/file: `pipelex validate path_to_file.plx`+- For all pipelines: `pipelex validate all`+- Remember: Ensure your Python virtual environment is activated (typically `.venv` for standard installations) before running `pipelex` commands.+Then, create an example file to run the pipeline in the `examples` folder.But don't write documentation unless asked explicitly to.diff --git a/pipelex/pipe_controllers/sub_pipe.py b/pipelex/pipe_controllers/sub_pipe.pyindex 14bc4a2cd..4e85b9fac 100644--- a/pipelex/pipe_controllers/sub_pipe.py+++ b/pipelex/pipe_controllers/sub_pipe.py@@ -6,12 +6,11 @@ from pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicityfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.exceptions import PipeInputError, WorkingMemoryStuffNotFoundError-from pipelex.hub import get_pipe_router, get_pipeline_tracker, get_required_pipe+from pipelex.hub import get_pipeline_tracker, get_required_pipefrom pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprintfrom pipelex.pipe_controllers.batch.pipe_batch_factory import PipeBatchFactoryfrom pipelex.pipe_controllers.condition.pipe_condition import PipeCondition-from pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunMode, PipeRunParams+from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunParamsfrom pipelex.pipeline.job_metadata import JobMetadata@@ -67,45 +66,20 @@ class SubPipe(BaseModel):blueprint=pipe_batch_blueprint,concept_codes_from_the_same_domain=self.concept_codes_from_the_same_domain,)- # This is the only line that changes between run and dry_run- if sub_pipe_run_params.run_mode == PipeRunMode.DRY:- sub_pipe_run_params.run_mode = PipeRunMode.DRY- pipe_output = await pipe_batch.run_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=sub_pipe_run_params,- output_name=self.output_name,- )- else:- sub_pipe_run_params.run_mode = PipeRunMode.LIVE- pipe_output = await pipe_batch.run_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=sub_pipe_run_params,- output_name=self.output_name,- )+ pipe_output = await pipe_batch.run_pipe(+ job_metadata=job_metadata,+ working_memory=working_memory,+ pipe_run_params=sub_pipe_run_params,+ output_name=self.output_name,+ )# Case 2: Condition processingelif isinstance(sub_pipe, PipeCondition):- # This is the only line that changes between run and dry_run- if sub_pipe_run_params.run_mode == PipeRunMode.DRY:- sub_pipe_run_params.run_mode = PipeRunMode.DRY- pipe_output = await sub_pipe.run_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=sub_pipe_run_params,- output_name=self.output_name,- )- else:- sub_pipe_run_params.run_mode = PipeRunMode.LIVE- pipe_output = await get_pipe_router().run(- pipe_job=PipeJobFactory.make_pipe_job(- pipe=get_required_pipe(pipe_code=self.pipe_code),- job_metadata=job_metadata,- working_memory=working_memory,- output_name=self.output_name,- pipe_run_params=sub_pipe_run_params,- ),- )+ pipe_output = await sub_pipe.run_pipe(+ job_metadata=job_metadata,+ working_memory=working_memory,+ pipe_run_params=sub_pipe_run_params,+ output_name=self.output_name,+ )else:# Case 3: Normal processingrequired_variables = sub_pipe.required_variables()@@ -119,33 +93,19 @@ class SubPipe(BaseModel):msg = f"Some required stuff(s) not found: {error_details}"raise PipeInputError(message=msg, pipe_code=self.pipe_code, variable_name=exc.variable_name, concept_code=None) from exclog.verbose(required_stuffs, title=f"Required stuffs for {self.pipe_code}")- # This is the only line that changes between run and dry_run-- if sub_pipe_run_params.run_mode == PipeRunMode.DRY:- pipe_output = await sub_pipe.run_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=sub_pipe_run_params,- output_name=self.output_name,- )- else:- sub_pipe_run_params.run_mode = PipeRunMode.LIVE- pipe_output = await get_pipe_router().run(- pipe_job=PipeJobFactory.make_pipe_job(- pipe=get_required_pipe(pipe_code=self.pipe_code),- job_metadata=job_metadata,- working_memory=working_memory,- output_name=self.output_name,- pipe_run_params=sub_pipe_run_params,- ),- )- new_output_stuff = pipe_output.main_stuff- for stuff in required_stuffs:- get_pipeline_tracker().add_pipe_step(- from_stuff=stuff,- to_stuff=new_output_stuff,- pipe_code=self.pipe_code,- pipe_layer=sub_pipe_run_params.pipe_layers,- comment="SubPipe on required_stuff",- )+ pipe_output = await sub_pipe.run_pipe(+ job_metadata=job_metadata,+ working_memory=working_memory,+ pipe_run_params=sub_pipe_run_params,+ output_name=self.output_name,+ )+ if new_output_stuff := pipe_output.working_memory.get_optional_main_stuff():+ for stuff in required_stuffs:+ get_pipeline_tracker().add_pipe_step(+ from_stuff=stuff,+ to_stuff=new_output_stuff,+ pipe_code=self.pipe_code,+ pipe_layer=sub_pipe_run_params.pipe_layers,+ comment="SubPipe on required_stuff",+ )return pipe_output