Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
remove migrate
thomashebrardee69214d6b77
diff --git a/docs/pages/migration/index.md b/docs/pages/migration/index.mddeleted file mode 100644index eb8bb995b..000000000--- a/docs/pages/migration/index.md+++ /dev/null@@ -1,265 +0,0 @@-# Migration Guide--This document provides guidance for migrating between different versions of Pipelex that introduce breaking changes.--## Concepts Migration--### Breaking Change--We changed the syntax for defining concepts in PLX library files. The key name for concept definitions has changed from `Concept` to `description`.--### What Changed--**Old syntax:**-```plx-[concept.MyConceptName]-Concept = "Description of the concept"-refines = "Text"-```--**New syntax:**-```plx-[concept.MyConceptName]-description = "Description of the concept"-refines = "Text"-```--### Why This Change--This change improves consistency in our PLX schema by:--- Using more descriptive field names (`description` instead of `Concept`)-- Aligning with the internal `ConceptBlueprint` data structure-- Making the PLX files more self-documenting and intuitive--### Migration Process--#### Automatic Migration--Use the built-in migration command to automatically update your PLX files:--```bash-# Preview changes without applying them-pipelex migrate run --dry-run--# Apply the migration to all PLX files-pipelex migrate run-```--The migration command will:--- Find all `.toml` files in your configured pipelines directory-- Replace `Concept =` with `description =` in concept definitions-- Preserve all other formatting and comments-- Create backups of modified files (with `.backup` extension)--#### Manual Migration--If you prefer to migrate manually or need to handle special cases:--1. **Locate your PLX files**: Find all pipeline library files (typically in your configured pipelines directory)-2. **Update concept definitions**: Change `Concept =` to `description =` in all `[concept.ConceptName]` sections-3. **Validate syntax**: Run `pipelex validate all -c path/to/your/pipelex/config/folder -c ./pipelex_libraries` to ensure your files are correctly formatted--### Examples--#### Simple Concept Definition--**Before:**-```plx-[concept.Article]-Concept = "A written composition on a specific topic"-refines = "Text"-```--**After:**-```plx-[concept.Article]-description = "A written composition on a specific topic"-refines = "Text"-```--#### Complex Concept with Structure--**Before:**-```plx-[concept.Photo]-Concept = "Photo"-structure = "ImageContent"-refines = "Image"-```--**After:**-```plx-[concept.Photo]-description = "Photo"-structure = "ImageContent"-refines = "Image"-```--### Validation--After migration, validate your pipeline files:--```bash-# Validate all pipeline files-pipelex validate all -c path/to/your/pipelex/config/folder--# Run a specific pipeline to test-# Pipeline execution is done through Python code, not CLI (coming soon)-```--### Troubleshooting--**Error: "ConceptBlueprint validation failed"**--- Check that all `Concept =` entries have been changed to `description =`-- Ensure no typos were introduced during migration-- Verify PLX syntax is still valid--**Error: "Failed to load PLX file"**--- Check file permissions-- Ensure PLX syntax is valid (no missing quotes, brackets, etc.)-- Review any custom modifications that might conflict--### Rollback--If you need to rollback the migration:--1. Stop using the new version of Pipelex-2. Restore from the `.backup` files created during migration-3. Or manually change `description =` back to `Concept =`--## Pipes Migration--### Breaking Change--We changed the syntax for defining pipes in PLX library files. The pipe definition format has been restructured to use explicit `type` and `description` fields instead of the implicit `PipeClassName = "description"` format.--### What Changed--**Old syntax:**-```plx-[pipe.my_pipe_name]-SomePipeClass = "Description of what this pipe does"-```--**New syntax:**-```plx-[pipe.my_pipe_name]-type = "SomePipeClass"-description = "Description of what this pipe does"-```--### Why This Change--This change improves consistency and clarity by:--- Making the pipe class type explicit and separate from its description-- Aligning pipe definitions with concept definitions (both use `description`)-- Improving readability and maintainability of PLX files-- Making it easier to parse and validate pipe configurations--### Migration Process--#### Automatic Migration--Use the built-in migration command to automatically update your PLX files:--```bash-# Preview changes without applying them-pipelex migrate run --dry-run--# Apply the migration to all PLX files-pipelex migrate run-```--The migration command will:--- Find all `.toml` files in your configured pipelines directory-- Convert `PipeClassName = "description"` to `type = "PipeClassName"` and `description = "description"`-- Preserve all other formatting and comments-- Create backups of modified files (with `.backup` extension)--#### Manual Migration--If you prefer to migrate manually:--1. **Locate your PLX files**: Find all pipeline library files-2. **Update pipe definitions**:- - Find all `[pipe.pipe_name]` sections- - Replace `PipeClassName = "description"` with:- - `type = "PipeClassName"`- - `description = "description"`-3. **Validate syntax**: Run `pipelex validate all -c path/to/your/pipelex/config/folder -c ./pipelex_libraries` to ensure your files are correctly formatted--### Examples--#### Simple Pipe Definition--**Before:**-```plx-[pipe.summarize_article]-TextSummarizer = "Summarize the given article into key points"-```--**After:**-```plx-[pipe.summarize_article]-type = "TextSummarizer"-description = "Summarize the given article into key points"-```--#### Pipe with Additional Configuration--**Before:**-```plx-[pipe.analyze_sentiment]-SentimentAnalyzer = "Analyze the sentiment of the input text"-model = "bert-base-uncased"-threshold = 0.8-```--**After:**-```plx-[pipe.analyze_sentiment]-type = "SentimentAnalyzer"-description = "Analyze the sentiment of the input text"-model = "bert-base-uncased"-threshold = 0.8-```--### Deprecation Warning--In the transition period, the old syntax will still work but will generate deprecation warnings:--```-WARNING: Pipe 'my_pipe_name' uses deprecated syntax. Please migrate to new format:-replace this syntax:-```-SomePipeClass = "Description"-```-by this:-```-type = "SomePipeClass"-description = "Description"-```-Old syntax will be removed in v0.3.0.-```--### Validation--After migration, validate your pipeline files:--```bash-# Validate all pipeline files-pipelex validate all -c path/to/your/pipelex/config/folder--# Run a specific pipeline to test-# Pipeline execution is done through Python code, not CLI (coming soon)-```-------For additional help or if you encounter issues during migration, please [open an issue](https://github.com/Pipelex/pipelex/issues) on our GitHub repository.\ No newline at end of filediff --git a/pipelex/cli/_cli.py b/pipelex/cli/_cli.pyindex b7dc649dd..85b041cf0 100644--- a/pipelex/cli/_cli.py+++ b/pipelex/cli/_cli.py@@ -3,7 +3,7 @@ from click import Command, Contextfrom typer.core import TyperGroupfrom typing_extensions import override-from pipelex.cli.commands import init_app, migrate_app, show_app, validate_app+from pipelex.cli.commands import init_app, show_app, validate_appfrom pipelex.cli.commands.build_cmd import build_app@@ -34,5 +34,4 @@ app = typer.Typer(app.add_typer(init_app, name="init", help="Initialization commands")app.add_typer(validate_app, name="validate", help="Validation and dry-run commands")app.add_typer(show_app, name="show", help="Show and list commands")-app.add_typer(migrate_app, name="migrate", help="Migration commands")app.add_typer(build_app, name="build", help="Build artifacts like pipeline blueprints")diff --git a/pipelex/cli/commands/__init__.py b/pipelex/cli/commands/__init__.pyindex 19b9cd5a6..fbec641e0 100644--- a/pipelex/cli/commands/__init__.py+++ b/pipelex/cli/commands/__init__.py@@ -4,8 +4,7 @@ This package organizes CLI commands into logical modules."""from .init_cmd import init_app-from .migrate_cmd import migrate_appfrom .show_cmd import show_appfrom .validate_cmd import validate_app-__all__ = ["init_app", "migrate_app", "show_app", "validate_app"]+__all__ = ["init_app", "show_app", "validate_app"]diff --git a/pipelex/cli/commands/migrate_cmd.py b/pipelex/cli/commands/migrate_cmd.pydeleted file mode 100644index cb1733872..000000000--- a/pipelex/cli/commands/migrate_cmd.py+++ /dev/null@@ -1,123 +0,0 @@-from __future__ import annotations--from pathlib import Path-from typing import TYPE_CHECKING, Annotated--import typer--from pipelex.cli.commands.common import is_pipelex_libraries_folder-from pipelex.migration.migrate_v0_1_0_to_v0_2_0 import TOMLMigrator, migrate_concept_syntax-from pipelex.migration.migration_result import MigrationResult--if TYPE_CHECKING:- from pipelex.migration.migration_result import MigrationResult---def do_migrate(- target_path: str = "./pipelex_libraries",- dry_run: bool = False,- backups: bool = True,-) -> None:- """Migrate TOML files to new syntax (Concept = -> description = and PipeClassName = -> type/description).-- The path can be:- - A Pipelex libraries folder (we'll use its `pipelines` subfolder)- - A directory containing TOML files- - A single TOML file- """- config_path = Path(target_path)-- if is_pipelex_libraries_folder(target_path):- pipelines_dir_or_file = config_path / "pipelines"- else:- pipelines_dir_or_file = config_path-- if not pipelines_dir_or_file.exists():- typer.echo(f"❌ Path not found at '{pipelines_dir_or_file}'")- raise typer.Exit(1)-- try:- result: MigrationResult = migrate_concept_syntax(- directory=pipelines_dir_or_file,- create_backups=backups and not dry_run,- dry_run=dry_run,- )-- for error in result.errors:- typer.echo(f"❌ {error}")-- if result.errors and not result.files_modified:- typer.echo("❌ Migration failed due to errors")- raise typer.Exit(1)-- typer.echo(f"Found {result.files_processed} TOML file(s) to check")-- if result.files_modified == 0:- typer.echo("✅ All TOML files are already using the new syntax")- return-- if dry_run:- migrator = TOMLMigrator()- # For display, if a single file path was provided, show relative to its parent- base_for_print = pipelines_dir_or_file if pipelines_dir_or_file.is_dir() else pipelines_dir_or_file.parent- for file_path in result.modified_files:- try:- with open(file_path, encoding="utf-8") as f:- content = f.read()-- changes = migrator.get_migration_preview(content)- try:- rel = file_path.relative_to(base_for_print)- except Exception:- rel = Path(file_path.name)- typer.echo(f"\n📄 {rel}:")- for change in changes:- typer.echo(f" Line {change['line_number']}: {change['old_line']} → {change['new_line']}")- except Exception as exc:- typer.echo(f"❌ Error reading {file_path}: {exc}")-- typer.echo(f"\n📋 Summary: Found {result.total_changes} change(s) in {result.files_modified} file(s)")- typer.echo(" Run without --dry-run to apply these changes")- else:- create_backups = backups and not dry_run- base_for_print = pipelines_dir_or_file if pipelines_dir_or_file.is_dir() else pipelines_dir_or_file.parent- for file_path in result.modified_files:- try:- rel = file_path.relative_to(base_for_print)- except Exception:- rel = Path(file_path.name)- typer.echo(f"✅ Migrated {rel}")- if create_backups:- backup_path = file_path.with_suffix(".toml.backup")- typer.echo(f" Backup saved to {backup_path.name}")-- typer.echo(f"\n✅ Migration completed: {result.total_changes} change(s) applied to {result.files_modified} file(s)")- if create_backups:- typer.echo(" Backup files created with .backup extension")- typer.echo(" Run 'pipelex validate all -c pipelex/libraries' to verify the migration")- except FileNotFoundError as exc:- typer.echo(f"❌ {exc}")- raise typer.Exit(1) from exc- except Exception as exc:- typer.echo(f"❌ Migration failed: {exc}")- raise typer.Exit(1) from exc---# Typer group for migration commands-migrate_app = typer.Typer(help="Migration commands", no_args_is_help=True)---@migrate_app.command("run")-def migrate_cmd(- target_path: Annotated[- str,- typer.Option(- "--path",- "-p",- help=("Path to Pipelex libraries folder (uses its pipelines), a directory of TOML files, or a single TOML file"),- ),- ] = "./pipelex_libraries",- dry_run: Annotated[bool, typer.Option("--dry-run", help="Preview changes without applying them")] = False,- backups: Annotated[bool, typer.Option("--backups/--no-backups", help="Create backup files before migration")] = False,-) -> None:- do_migrate(target_path=target_path, dry_run=dry_run, backups=backups)