Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Feature/small tweaks (#557)

Louis Choquel3ef97715c5c8
diff --git a/.blackboxrules b/.blackboxrules
index 402bd7140..b34c89a60 100644
--- a/.blackboxrules
+++ b/.blackboxrules
@@ -20,6 +20,14 @@
Always fix any issues reported by these tools before proceeding.
+### Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
### Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
@@ -139,7 +147,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+#### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+#### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
@@ -254,6 +271,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/.cursor/rules/commands.mdc b/.cursor/rules/commands.mdc
index 48817b5fa..2db239c35 100644
--- a/.cursor/rules/commands.mdc
+++ b/.cursor/rules/commands.mdc
@@ -21,6 +21,14 @@ description: Guidelines for running commands
Always fix any issues reported by these tools before proceeding.
+## Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
## Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
diff --git a/.cursor/rules/pytest_standards.mdc b/.cursor/rules/pytest_standards.mdc
index 3c5d361b8..c597b959b 100644
--- a/.cursor/rules/pytest_standards.mdc
+++ b/.cursor/rules/pytest_standards.mdc
@@ -18,6 +18,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/.cursor/rules/python_standards.mdc b/.cursor/rules/python_standards.mdc
index 37bef5c88..03c24d7d9 100644
--- a/.cursor/rules/python_standards.mdc
+++ b/.cursor/rules/python_standards.mdc
@@ -36,7 +36,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index 402bd7140..b34c89a60 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -20,6 +20,14 @@
Always fix any issues reported by these tools before proceeding.
+### Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
### Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
@@ -139,7 +147,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+#### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+#### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
@@ -254,6 +271,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/.pipelex/inference/backends/internal.toml b/.pipelex/inference/backends/internal.toml
index 2d30b7fe0..e44b2227f 100644
--- a/.pipelex/inference/backends/internal.toml
+++ b/.pipelex/inference/backends/internal.toml
@@ -19,7 +19,7 @@
################################################################################
# --- PyPDFium2 Text Extractor -------------------------------------------------
-[pypdfium2-extract-text]
+[pypdfium2-extract-pdf]
model_type = "text_extractor"
sdk = "pypdfium2"
model_id = "extract-text"
diff --git a/.pipelex/inference/deck/base_deck.toml b/.pipelex/inference/deck/base_deck.toml
index bb28c3a96..c194c36ad 100644
--- a/.pipelex/inference/deck/base_deck.toml
+++ b/.pipelex/inference/deck/base_deck.toml
@@ -111,7 +111,7 @@ cheap_llm_for_creativity = [
pdf_text_extractor = [
"azure-document-intelligence",
"mistral-ocr",
- "pypdfium2-extract-text",
+ "pypdfium2-extract-pdf",
]
image_text_extractor = ["mistral-ocr"]
@@ -173,11 +173,11 @@ for_object = "cheap_llm_for_structured"
####################################################################################################
[extract]
-choice_default = "extract_text_from_visuals"
+choice_default = "extract_ocr_from_document"
[extract.presets]
-extract_text_from_visuals = { model = "azure-document-intelligence", max_nb_images = 100, image_min_size = 50 }
-extract_text_from_pdf = { model = "pypdfium2-extract-text", max_nb_images = 100, image_min_size = 50 }
+extract_ocr_from_document = { model = "azure-document-intelligence", max_nb_images = 100, image_min_size = 50 }
+extract_basic_from_pdf = { model = "pypdfium2-extract-pdf", max_nb_images = 100, image_min_size = 50 }
####################################################################################################
# Image Generation Presets
diff --git a/.windsurfrules.md b/.windsurfrules.md
index 402bd7140..b34c89a60 100644
--- a/.windsurfrules.md
+++ b/.windsurfrules.md
@@ -20,6 +20,14 @@
Always fix any issues reported by these tools before proceeding.
+### Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
### Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
@@ -139,7 +147,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+#### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+#### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
@@ -254,6 +271,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/AGENTS.md b/AGENTS.md
index 051c86aee..dbdeb9260 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -20,6 +20,14 @@
Always fix any issues reported by these tools before proceeding.
+### Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
### Running Tests in Codex Cloud
To test everything that can be tested from within the Codex Cloud sandbox, run this:
@@ -96,7 +104,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+#### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+#### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
@@ -211,6 +228,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/CLAUDE.md b/CLAUDE.md
index 402bd7140..b34c89a60 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -20,6 +20,14 @@
Always fix any issues reported by these tools before proceeding.
+### Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
### Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
@@ -139,7 +147,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+#### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+#### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
@@ -254,6 +271,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/README.md b/README.md
index 1602c65dd..2e71d6e0f 100644
--- a/README.md
+++ b/README.md
@@ -119,7 +119,7 @@ Extracts text content from the candidate's CV PDF document using OCR technology,
"""
inputs = { cv_pdf = "PDF" }
output = "Page[]"
-model = "extract_text_from_pdf"
+model = "extract_basic_from_pdf"
[pipe.extract_job_offer_text]
type = "PipeExtract"
@@ -128,7 +128,7 @@ Extracts text content from the job offer PDF document using OCR technology, conv
"""
inputs = { job_offer_pdf = "PDF" }
output = "Page[]"
-model = "extract_text_from_pdf"
+model = "extract_basic_from_pdf"
[pipe.analyze_match]
type = "PipeLLM"
diff --git a/docs/home/3-understand-pipelex/language-spec-v0-1-0.md b/docs/home/3-understand-pipelex/language-spec-v0-1-0.md
index eb841795d..662aac1a6 100644
--- a/docs/home/3-understand-pipelex/language-spec-v0-1-0.md
+++ b/docs/home/3-understand-pipelex/language-spec-v0-1-0.md
@@ -98,7 +98,7 @@ type = "PipeExtract"
description = "Extracts text content from the job offer PDF document into pages."
inputs = { job_offer_pdf = "PDF" }
output = "Page[]"
-model = "extract_text_from_pdf"
+model = "extract_basic_from_pdf"
[pipe.structure_job_requirements]
type = "PipeLLM"
@@ -143,7 +143,7 @@ type = "PipeExtract"
description = "Extracts text content from a single CV PDF into pages."
inputs = { cv_pdf = "PDF" }
output = "Page[]"
-model = "extract_text_from_pdf"
+model = "extract_basic_from_pdf"
[pipe.analyze_cv_match]
type = "PipeLLM"
diff --git a/docs/home/6-build-reliable-ai-workflows/pipes/pipe-operators/PipeExtract.md b/docs/home/6-build-reliable-ai-workflows/pipes/pipe-operators/PipeExtract.md
index 41bec8749..0dc59450f 100644
--- a/docs/home/6-build-reliable-ai-workflows/pipes/pipe-operators/PipeExtract.md
+++ b/docs/home/6-build-reliable-ai-workflows/pipes/pipe-operators/PipeExtract.md
@@ -33,7 +33,7 @@ PipeExtract uses the unified inference backend system to manage OCR models. This
Common OCR model handles:
- `mistral-ocr`: Mistral's OCR model for high-quality text and image extraction
-- `pypdfium2-extract-text`: Local PDF text extraction (no API calls required)
+- `pypdfium2-extract-pdf`: Local PDF text extraction (no API calls required)
OCR presets are defined in your model deck configuration and can include parameters like `max_nb_images` and `image_min_size`.
@@ -49,7 +49,7 @@ OCR presets are defined in your model deck configuration and can include paramet
| `page_views` | boolean | If `true`, a high-fidelity image of each page will be included in the `page_view` field. Defaults to `false`. | No |
| `page_views_dpi` | integer | The resolution (in Dots Per Inch) for the generated page views when processing a PDF. Defaults to `150`. | No |
| `page_image_captions` | boolean | If `true`, the OCR service may attempt to generate captions for the images found. *Note: This feature depends on the OCR provider.* | No |
-| `model` | string | The Extract model choice by name, setting, or preset to use (e.g., `"mistral-ocr"`, `"extract_text_from_visuals"`). Defaults to the model specified in the global config. | No |
+| `model` | string | The Extract model choice by name, setting, or preset to use (e.g., `"mistral-ocr"`, `"extract_ocr_from_document"`). Defaults to the model specified in the global config. | No |
### Example: Processing a PDF
diff --git a/docs/home/7-configuration/config-technical/inference-backend-config.md b/docs/home/7-configuration/config-technical/inference-backend-config.md
index 3f7c700d0..08fc1dc84 100644
--- a/docs/home/7-configuration/config-technical/inference-backend-config.md
+++ b/docs/home/7-configuration/config-technical/inference-backend-config.md
@@ -452,8 +452,8 @@ OCR presets combine OCR model selection with optimized parameters:
```toml
[extract.presets]
# General purpose OCR
-extract_text_from_visuals = { ocr_handle = "mistral-ocr", max_nb_images = 100, image_min_size = 50 }
-extract_text_from_pdf = { model = "pypdfium2-extract-text", max_nb_images = 100, image_min_size = 50 }
+extract_ocr_from_document = { ocr_handle = "mistral-ocr", max_nb_images = 100, image_min_size = 50 }
+extract_basic_from_pdf = { model = "pypdfium2-extract-pdf", max_nb_images = 100, image_min_size = 50 }
```
### Image Generation Presets
@@ -478,7 +478,7 @@ for_text = "cheap_llm_for_text"
for_object = "cheap_llm_for_object"
[extract]
-choice_default = "extract_text_from_visuals"
+choice_default = "extract_ocr_from_document"
[img_gen]
choice_default = "gen_image_basic"
@@ -504,7 +504,7 @@ my_custom_img_gen = { model = "flux-dev", quality = "medium" }
# Add custom aliases
[aliases]
my_custom_llm = "claude-3-sonnet"
-my_custom_extract = "pypdfium2-extract-text"
+my_custom_extract = "pypdfium2-extract-pdf"
my_custom_img_gen = "base-img-gen"
```
diff --git a/pipelex/kit/agent_rules/codex_commands.md b/pipelex/kit/agent_rules/codex_commands.md
index d0b45a1ea..6557a4ee7 100644
--- a/pipelex/kit/agent_rules/codex_commands.md
+++ b/pipelex/kit/agent_rules/codex_commands.md
@@ -17,6 +17,14 @@
Always fix any issues reported by these tools before proceeding.
+## Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
## Running Tests in Codex Cloud
To test everything that can be tested from within the Codex Cloud sandbox, run this:
diff --git a/pipelex/kit/agent_rules/commands.md b/pipelex/kit/agent_rules/commands.md
index a3da0a706..75b65e0ae 100644
--- a/pipelex/kit/agent_rules/commands.md
+++ b/pipelex/kit/agent_rules/commands.md
@@ -17,6 +17,14 @@
Always fix any issues reported by these tools before proceeding.
+## Cleaning Derived Files
+
+ If you need to clean derived files and caches, typically after you erased files or moved tests, the linters can get confused, the pytest collection can be off...
+
+ ```bash
+ make cleanderived
+ ```
+
## Running Tests
After you're finished making code changes, you must always run tests using `make test-xdist`.
diff --git a/pipelex/kit/agent_rules/pytest_standards.md b/pipelex/kit/agent_rules/pytest_standards.md
index 6834d125a..7e1ff93c8 100644
--- a/pipelex/kit/agent_rules/pytest_standards.md
+++ b/pipelex/kit/agent_rules/pytest_standards.md
@@ -12,6 +12,7 @@ NEVER USE unittest.mock. YOU MUST USE pytest-mock instead.
- `tests/integration/` - for integration tests that test component interactions
- `tests/e2e/` - for end-to-end tests that test complete workflows
- `tests/test_pipelines/` - for test pipeline definitions (PLX files and their structuring python files)
+- Do NOT add `__init__.py` files to test directories. Test directories do not need to be Python packages.
- Fixtures are defined in conftest.py modules at different levels of the hierarchy, their scope is handled by pytest
- Test data is placed inside test_data.py at different levels of the hierarchy, they must be imported with package paths from the root like `from tests.integration.pipelex.cogt.test_data`. Their content is all constants, regrouped inside classes to keep things tidy.
- Always put test inside Test classes: 1 TestClass per module.
diff --git a/pipelex/kit/agent_rules/python_standards.md b/pipelex/kit/agent_rules/python_standards.md
index 32c6fa9c3..855f51972 100644
--- a/pipelex/kit/agent_rules/python_standards.md
+++ b/pipelex/kit/agent_rules/python_standards.md
@@ -30,7 +30,16 @@ This document outlines the core coding standards, best practices, and quality co
- Import all necessary libraries at the top of the file
- Do not import libraries in functions or classes unless in very specific cases, to be discussed with the user, as they would required a `# noqa: ...` comment to pass linting
- - Do not bother with ordering the imports, our Ruff linter will handle it for us. Same goes with removing unused imports.
+ - Do not bother with ordering the imports, our Ruff linter will handle it for us.
+
+### **Removing unused imports**
+
+ - To remove unused imports, run `make fix-unused-imports` or `make fui` (shorthand). This is faster and cheaper than rewriting with LLM.
+
+### **No re-exports in `__init__.py`**
+
+ - Do NOT fill `__init__.py` files with re-exports.
+ - Always use direct full-path imports everywhere. For example:
- **Logging and Pretty Printing**:
diff --git a/pipelex/kit/configs/inference/backends/internal.toml b/pipelex/kit/configs/inference/backends/internal.toml
index 2d30b7fe0..e44b2227f 100644
--- a/pipelex/kit/configs/inference/backends/internal.toml
+++ b/pipelex/kit/configs/inference/backends/internal.toml
@@ -19,7 +19,7 @@
################################################################################
# --- PyPDFium2 Text Extractor -------------------------------------------------
-[pypdfium2-extract-text]
+[pypdfium2-extract-pdf]
model_type = "text_extractor"
sdk = "pypdfium2"
model_id = "extract-text"
diff --git a/pipelex/kit/configs/inference/deck/base_deck.toml b/pipelex/kit/configs/inference/deck/base_deck.toml
index bb28c3a96..c194c36ad 100644
--- a/pipelex/kit/configs/inference/deck/base_deck.toml
+++ b/pipelex/kit/configs/inference/deck/base_deck.toml
@@ -111,7 +111,7 @@ cheap_llm_for_creativity = [
pdf_text_extractor = [
"azure-document-intelligence",
"mistral-ocr",
- "pypdfium2-extract-text",
+ "pypdfium2-extract-pdf",
]
image_text_extractor = ["mistral-ocr"]
@@ -173,11 +173,11 @@ for_object = "cheap_llm_for_structured"
####################################################################################################
[extract]
-choice_default = "extract_text_from_visuals"
+choice_default = "extract_ocr_from_document"
[extract.presets]
-extract_text_from_visuals = { model = "azure-document-intelligence", max_nb_images = 100, image_min_size = 50 }
-extract_text_from_pdf = { model = "pypdfium2-extract-text", max_nb_images = 100, image_min_size = 50 }
+extract_ocr_from_document = { model = "azure-document-intelligence", max_nb_images = 100, image_min_size = 50 }
+extract_basic_from_pdf = { model = "pypdfium2-extract-pdf", max_nb_images = 100, image_min_size = 50 }
####################################################################################################
# Image Generation Presets
diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py
index c01f19603..40be993ea 100644
--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py
+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py
@@ -14,7 +14,6 @@ from pipelex.pipeline.execute import execute_pipeline
@pytest.mark.inference
@pytest.mark.asyncio
class TestPipeSequenceMultiplicity:
- @pytest.mark.usefixtures("request")
async def test_creative_ideation_sequence_with_multiplicity(self, pipe_run_mode: PipeRunMode):
"""Test creative ideation sequence with nb_output and batching."""
# Create test input
diff --git a/tests/integration/pipelex/cogt/test_content_generator.py b/tests/integration/pipelex/cogt/test_content_generator.py
index 0b082dd0f..ac9bbf4a7 100644
--- a/tests/integration/pipelex/cogt/test_content_generator.py
+++ b/tests/integration/pipelex/cogt/test_content_generator.py
@@ -114,7 +114,6 @@ class TestContentGenerator:
pretty_print(image, title=f"ContentGenerator: Image generated by '{img_gen_handle}' for '{positive_text}'")
assert isinstance(image, ImageContent)
- @pytest.mark.usefixtures("request")
async def test_make_templated_text(self, content_generator: ContentGeneratorProtocol):
context = {
"the_answer": "elementary, my dear Watson",
diff --git a/tests/integration/pipelex/fixtures/extract_fixtures.py b/tests/integration/pipelex/fixtures/extract_fixtures.py
index dd877334d..57e419d50 100644
--- a/tests/integration/pipelex/fixtures/extract_fixtures.py
+++ b/tests/integration/pipelex/fixtures/extract_fixtures.py
@@ -24,7 +24,7 @@ def is_extract_handle_supported_by_enabled_backends(extract_handle: str) -> bool
# ================================================================================================
EXTRACT_HANDLE_FROM_PDF = [
- "pypdfium2-extract-text",
+ "pypdfium2-extract-pdf",
"docling-extract-text",
"mistral-ocr",
"mistral-ocr-2503",
@@ -86,8 +86,8 @@ def extract_handle_from_image(request: pytest.FixtureRequest) -> str:
@pytest.fixture(
params=[
- "extract_text_from_visuals",
- "extract_text_from_pdf",
+ "extract_ocr_from_document",
+ "extract_basic_from_pdf",
],
)
def extract_choice_for_pdf(request: pytest.FixtureRequest) -> str:
@@ -97,7 +97,7 @@ def extract_choice_for_pdf(request: pytest.FixtureRequest) -> str:
@pytest.fixture(
params=[
- "extract_text_from_visuals",
+ "extract_ocr_from_document",
],
)
def extract_choice_for_image(request: pytest.FixtureRequest) -> str:
diff --git a/tests/integration/pipelex/pipes/test_image_inputs.py b/tests/integration/pipelex/pipes/test_image_inputs.py
index 0e9073460..8908b7988 100644
--- a/tests/integration/pipelex/pipes/test_image_inputs.py
+++ b/tests/integration/pipelex/pipes/test_image_inputs.py
@@ -111,7 +111,6 @@ class TestImageInputs:
assert pipe_output.working_memory is not None
assert pipe_output.main_stuff is not None
- @pytest.mark.usefixtures("request")
async def test_image_input_within_concept_with_text(self, load_test_library: Callable[[list[Path]], None]) -> None:
load_test_library([Path("tests/integration/pipelex/pipes/pipelines")])
pipe_llm_blueprint = PipeLLMBlueprint(
diff --git a/tests/unit/pipelex/cogt/models/test_model_deck.py b/tests/unit/pipelex/cogt/models/test_model_deck.py
index 8b317663f..be5b69397 100644
--- a/tests/unit/pipelex/cogt/models/test_model_deck.py
+++ b/tests/unit/pipelex/cogt/models/test_model_deck.py
@@ -40,7 +40,7 @@ class TestModelDeckGetOptionalInferenceModel:
for_object=LLMSetting(model="default_object", temperature=0.1, max_tokens=1000),
),
extract_presets={},
- extract_choice_default="extract_text_from_visuals",
+ extract_choice_default="extract_ocr_from_document",
img_gen_presets={},
img_gen_choice_default="gen_image_basic",
model_deck_config=ModelDeckConfig(is_model_fallback_enabled=is_model_fallback_enabled, missing_presets_reaction=ProblemReaction.NONE),
diff --git a/tests/unit/pipelex/core/memory/test_dry_working_memory.py b/tests/unit/pipelex/core/memory/test_dry_working_memory.py
index c31535ca7..234126eb4 100644
--- a/tests/unit/pipelex/core/memory/test_dry_working_memory.py
+++ b/tests/unit/pipelex/core/memory/test_dry_working_memory.py
@@ -13,10 +13,7 @@ from tests.unit.pipelex.core.memory.tricky_questions import ThoughtfulAnswer
@pytest.mark.dry_runnable
@pytest.mark.asyncio(loop_scope="class")
class TestDryWorkingMemory:
- @pytest.mark.usefixtures("request")
- async def test_make_for_dry_run_with_page_content(
- self,
- ):
+ async def test_make_for_dry_run_with_page_content(self):
log.info("Testing dry run with PageContent")
# Test the specific inputs requested by the user
@@ -54,10 +51,7 @@ class TestDryWorkingMemory:
# Verify page_view field exists (it's Optional so could be None)
assert hasattr(page_content, "page_view")
- @pytest.mark.usefixtures("request")
- async def test_make_for_dry_run_with_structured_content(
- self,
- ):
+ async def test_make_for_dry_run_with_structured_content(self):
log.info("Testing dry run with structured content (ThoughtfulAnswer)")
# Use ThoughtfulAnswer from tricky questions domain
@@ -112,10 +106,7 @@ class TestDryWorkingMemory:
log.info("Created mock working memory with structured content:")
dry_memory.pretty_print_summary()
- @pytest.mark.usefixtures("request")
- async def test_make_for_dry_run_with_text_content_fallback(
- self,
- ):
+ async def test_make_for_dry_run_with_text_content_fallback(self):
log.info("Testing dry run with TextContent fallback")
needed_inputs = [
@@ -160,10 +151,7 @@ class TestDryWorkingMemory:
log.info("Created mock working memory with TextContent fallback:")
dry_memory.pretty_print_summary()
- @pytest.mark.usefixtures("request")
- async def test_make_for_dry_run_mixed_content_types(
- self,
- ):
+ async def test_make_for_dry_run_mixed_content_types(self):
log.info("Testing dry run with mixed content types")
needed_inputs = [
@@ -218,10 +206,7 @@ class TestDryWorkingMemory:
log.info("Created mock working memory with mixed content types:")
dry_memory.pretty_print_summary()
- @pytest.mark.usefixtures("request")
- async def test_make_for_dry_run_empty_inputs(
- self,
- ):
+ async def test_make_for_dry_run_empty_inputs(self):
log.info("Testing dry run with empty inputs")
needed_inputs: list[TypedNamedStuffSpec] = []
@@ -234,7 +219,6 @@ class TestDryWorkingMemory:
log.info("Created empty mock working memory")
- @pytest.mark.usefixtures("request")
async def test_make_for_dry_run_realistic_pipeline_scenario(self):
log.info("Testing dry run with realistic tricky questions pipeline scenario")
diff --git a/tests/unit/pipelex/core/test_data/pipes/operators/extract/pipe_extract.py b/tests/unit/pipelex/core/test_data/pipes/operators/extract/pipe_extract.py
index 95309e12e..ab7d74077 100644
--- a/tests/unit/pipelex/core/test_data/pipes/operators/extract/pipe_extract.py
+++ b/tests/unit/pipelex/core/test_data/pipes/operators/extract/pipe_extract.py
@@ -12,7 +12,7 @@ type = "PipeExtract"
description = "Extract text from document"
inputs = { document = "PDF" }
output = "Page[]"
-model = "extract_text_from_pdf"
+model = "extract_basic_from_pdf"
""",
PipelexBundleBlueprint(
domain="test_pipes",
@@ -25,7 +25,7 @@ model = "extract_text_from_pdf"
"document": NativeConceptCode.PDF,
},
output=NativeConceptCode.PAGE.as_output_multiple_indeterminate,
- model="extract_text_from_pdf",
+ model="extract_basic_from_pdf",
),
},
),
diff --git a/tests/unit/pipelex/tools/typing/test_structure_printer.py b/tests/unit/pipelex/tools/typing/test_structure_printer.py
index aeee6bad2..3e1692278 100644
--- a/tests/unit/pipelex/tools/typing/test_structure_printer.py
+++ b/tests/unit/pipelex/tools/typing/test_structure_printer.py
@@ -1,7 +1,6 @@
from datetime import datetime
from typing import Literal
-import pytest
from pydantic import BaseModel, Field, field_validator
from pipelex import pretty_print
@@ -123,7 +122,6 @@ class GanttChart(StructuredContent):
class TestStructurePrinter:
- @pytest.mark.usefixtures("request")
def test_simple_text_content(self):
"""Test structure of simple text content"""
result = StructurePrinter().get_type_structure(SimpleTextContent)
@@ -134,7 +132,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_simple_structured_content(self):
"""Test structure of simple structured content"""
result = StructurePrinter().get_type_structure(SimpleStructuredContent)
@@ -147,7 +144,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_enum_content(self):
"""Test structure of content with enum"""
result = StructurePrinter().get_type_structure(DocumentTypeContent)
@@ -162,7 +158,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_nested_content(self):
"""Test structure of nested content"""
result = StructurePrinter().get_type_structure(PersonContent)
@@ -198,7 +193,6 @@ class TestStructurePrinter:
pretty_print(expected, "expected")
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_list_content(self):
"""Test structure of list content"""
result = StructurePrinter().get_type_structure(ComplexListContent)
@@ -235,7 +229,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_model_with_field_description(self):
"""Test structure of a model with field descriptions"""
@@ -257,7 +250,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_model_with_docstring_and_field_description(self):
"""Test structure of a model with both docstring and field descriptions"""
@@ -286,7 +278,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_literal_field_content(self) -> None:
"""Test structure of content with Literal field"""
result = StructurePrinter().get_type_structure(MusicCategoryContent)
@@ -311,7 +302,6 @@ class TestStructurePrinter:
]
assert result == expected, f"Expected:\n{''.join(expected)}\n\nGot:\n{''.join(result)}"
- @pytest.mark.usefixtures("request")
def test_gantt_chart_content(self):
result = StructurePrinter().get_type_structure(GanttChart, base_class=StructuredContent)
expected = [