Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Pin kajson to 0.4.2 and scope dynamic enums per-payload (#859)
Louis Choquel631125cbd452
diff --git a/.badges/tests.json b/.badges/tests.jsonindex 8cfc219b6..f718e693f 100644--- a/.badges/tests.json+++ b/.badges/tests.json@@ -1,7 +1,7 @@{"schemaVersion": 1,"label": "tests",- "message": "4194",+ "message": "4854","color": "blue","cacheSeconds": 300}\ No newline at end of filediff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9f4dd8edc..230acb044 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,24 +1,55 @@# Changelog-## [Unreleased]+## [v0.26.0] - 2026-05-04-### Added+### Highlights-- **`PipeOutput.graph_assembly_error`**: optional `str` field set when the Temporal `act_assemble_graph` activity raises so consumers can distinguish "graph never produced" from "graph assembly failed". Previously the failure was logged as a warning and `graph_spec` silently stayed `None`.-- **`SecurityError(PipelexError)`**: new base class in `pipelex.base_exceptions` for security-policy violations. `UnsafeSchemaError` now extends it (was `ContentGenerationError`) so security signals are not silently swallowed by domain `except` handlers.+- **Temporal distributed execution** — Pipelex pipelines can now run as Temporal workflows across worker processes. New `pipelex[temporal]` extra, `pipelex worker` CLI, `PipeRun` layer, and `LibraryCrate` IR for cross-process library propagation.+- **Distributed tracing** — pluggable event-log backends (NDJSON, DynamoDB, in-memory, buffering) emit cross-worker `TraceEvent`s that reassemble offline into a complete `GraphSpec`. Enabled by default; works in single-process mode too.++### Added-- **PipeRun layer**: New `PipeRunProtocol` with `PipeRun` (direct) and `TemporalPipeRun` (Temporal) implementations. Wraps pipe execution + delivery in a single orchestration unit. In Temporal mode, `WfPipeRun` workflow orchestrates `WfPipeRouter` as a child workflow followed by delivery activities.-- **Delivery framework**: `DeliveryAssignment` model with `WebhookDeliveryAssignment` (HTTP POST) and `StorageDeliveryAssignment` (S3/local). Deliveries run as Temporal activities with automatic retry, or inline in direct mode.-- **`TemporalPipeRun.start()`**: Non-blocking start that returns `workflow_id` + `WorkflowHandle` immediately. Replaces the former `PipeRouterTop.start_pipe_job()`.-- **Pre-generated `pipeline_run_id` support**: `pipeline_run_setup()`, `PipelineFactory`, and `PipelineManager` accept an optional `pipeline_run_id` parameter for external run record creation.+#### Temporal distributed execution++- **`pipelex[temporal]` extra and full Temporal integration package (`pipelex/temporal/`)**: includes `temporal_manager`, `temporal_connect`, `temporal_data_converter` (kajson-based JSON), `sandbox_manager`, `task_manager`, and a `pipelex worker` CLI subcommand that boots a worker against a configured queue. Toggle via `[temporal] is_enabled` in `pipelex.toml`. Bumps `temporalio` to `1.23.0`.+- **`PipeRun` layer (`pipelex/pipe_run/`)**: new `PipeRunProtocol` with `PipeRun` (in-process) and `TemporalPipeRun` (workflow-orchestrated) implementations. Wraps pipe execution + delivery in a single orchestration unit. In Temporal mode, `WfPipeRun` orchestrates `WfPipeRouter` as a child workflow followed by delivery activities. `TemporalPipeRun.start()` is non-blocking and returns `workflow_id` + `WorkflowHandle` immediately.+- **`TemporalPipeRouter`**: single router that auto-detects context (top-level vs child workflow) and dispatches accordingly.+- **Delivery framework**: `DeliveryAssignment` model with `WebhookDeliveryAssignment` (HTTP POST) and `StorageDeliveryAssignment` (S3 / local). Deliveries run as Temporal activities with automatic retry, or inline in direct mode.+- **`LibraryCrate` IR (`pipelex/libraries/library_crate.py`, `library_crate_factory.py`)**: serializable intermediate representation that lets a Temporal worker reconstruct the exact set of libraries (concepts, pipes, structures) required to execute a job — no shared filesystem assumed.+- **`StoragePayloadCodec`**: offloads Temporal payloads larger than a configurable threshold (default 1 MiB) to external storage (S3 or local) and replaces them with lightweight URI references. Configurable via `[temporal.payload_codec_config]`. New `pipelex codec serve` CLI launches the codec server consumed by the Temporal Web UI for payload inspection.+- **Schema-based dynamic model construction (`pipelex/cogt/content_generation/schema_to_model_factory.py`)**: rebuilds dynamic Pydantic models on a Temporal worker from the JSON schema embedded in `ObjectAssignment`, instead of relying on cross-process class identity. Cache bounded by an LRU (`_SCHEMA_CACHE_MAX_SIZE = 1024`); codegen serialized through a file lock to kill thundering-herd duplication on long-running workers receiving many distinct schemas. Adds `datamodel-code-generator>=0.55.0` as a runtime dependency.+- **Worker scopes**: `pipelex worker --scope ...` lets a worker subscribe to a subset of activities/workflows so deployments can specialize workers (e.g. inference workers vs orchestration workers).+- **Pre-generated `pipeline_run_id` support**: `pipeline_run_setup()`, `PipelineFactory`, and `PipelineManager` accept an optional `pipeline_run_id` parameter for external run-record creation.- **`workflow_id` in start response**: `PipelexPipelineStartResponse` includes `workflow_id`.- **Temporal `callbacks` passthrough**: `WorkflowExecutor.start_workflow()` accepts optional Temporal `Callback` objects.-- **DynamoDB event log backend**: New `DynamoDBEventLog` implementation of `EventLogProtocol` for cloud-based trace event storage. Installable via `pip install "pipelex[dynamodb]"`.-- **Event log DI**: Event log backend is now configurable via `TracingConfig.backend` (`"ndjson"` or `"dynamodb"`) and injectable via `Pipelex.setup(event_log=...)`. Factory function `make_event_log()` selects the backend from config or hub injection.-- **Hub event_log support**: `PipelexHub.set_event_log()` / `get_event_log()` for dependency injection of custom event log backends.++#### Distributed tracing (`pipelex/tracing/`)++- **`EventLogProtocol` with pluggable backends**: `NdjsonEventLog`, `DynamoDBEventLog`, `InMemoryEventLog`, `BufferingEventLog`. Emits `TraceEvent`s as pipes start/finish across processes. Configured via `[pipelex.tracing_config]` (`backend = "ndjson" | "dynamodb"`). Tracing is **enabled by default** and decoupled from Temporal — single-process runs also produce a trace.+- **`pipelex[dynamodb]` extra**: `pip install "pipelex[dynamodb]"` for the DynamoDB event log backend.+- **`writer_id` on `TraceEvent`**: every event records which writer (worker / process) produced it, propagated through every backend. Enables proper cross-worker dedup and per-writer aggregation.+- **`GraphSpecAssembler`**: rebuilds a complete `GraphSpec` offline from a stream of `TraceEvent`s — equivalence with the live `GraphTracer` is pinned by tests.+- **`UsageAggregator`**: aggregates per-pipeline LLM/extract/img-gen usage across workers from the event log.+- **Hub DI**: `PipelexHub.set_event_log()` / `get_event_log()` for dependency injection of custom event log backends. `make_event_log()` factory selects the backend from config or hub injection; the 3 prior call sites that hardcoded `NdjsonEventLog` now go through it.++#### Inference error classification++- **`InferenceErrorCategory` enum (`transient`, `configuration`, `content`, `capacity`)** with per-provider helpers that distinguish quota exhaustion from rate limits and detect content-policy violations — covers OpenAI, Anthropic, Google, Mistral, AWS Bedrock, and Pipelex Gateway.+- **All inference workers attach error category + user action**: every LLM, image-gen, extract, and search worker across all providers raises exceptions with an `InferenceErrorCategory` and actionable `user_action` hint.+- **Structured `ErrorReport`**: `PipelexError.to_error_report()` returns a dataclass with error type, category, retryable flag, user-action hint, model, and provider. CLI error handlers consume `to_error_report()` for consistent, structured display.+- **`SecurityError(PipelexError)`**: new base class in `pipelex.base_exceptions` for security-policy violations. `UnsafeSchemaError` now extends it (was `ContentGenerationError`) so security signals are not silently swallowed by domain `except` handlers.++#### Documentation++- New "Under the Hood" pages: `pipe-routing-and-execution.md`, `temporal-integration.md`, `distributed-content-generation.md`. Cover routing/dispatch, Temporal architecture, and the dynamic-model + storage-codec serialization story.+- New `AGENTS.md` at repo root, generated from the same source as `CLAUDE.md` via `make rules`.++#### Other++- **`PipeOutput.graph_assembly_error`**: optional `str` field set when the Temporal `act_assemble_graph` activity raises so consumers can distinguish "graph never produced" from "graph assembly failed". Previously the failure was logged as a warning and `graph_spec` silently stayed `None`.+- **`pipe extract` web URL support** with full test coverage in `tests/integration/pipelex/pipes/pipe_operators/pipe_extract/`.+- **`needs_inference` flag on CLI commands**: non-inference subcommands (e.g. `worker`, `doctor`) skip inference setup, shaving cold-start time.- **Environment-specific config**: `RunEnvironment` enum values updated to full names (`"development"`, `"production"`). Config loaded from `ENVIRONMENT` env var (was `ENV`).-- **Inference error classification**: New `InferenceErrorCategory` enum (`transient`, `configuration`, `content`, `capacity`) with per-provider helpers that distinguish quota exhaustion from rate limits and detect content policy violations — covers OpenAI, Anthropic, Google, Mistral, AWS Bedrock, and Gateway.-- **Structured `ErrorReport`**: `PipelexError.to_error_report()` returns a dataclass with error type, category, retryable flag, user action hint, model, and provider.### Changed@@ -26,40 +57,26 @@- Anthropic: bare `"credit"` token replaced with `"credit balance"`, `"out of credits"`, `"insufficient credit"`. No longer misclassifies `"your credit card was declined"` or `"we will credit your account"` as quota exhaustion.- Google: bare `"billing"` token replaced with `"billing limit"`, `"billing quota"`, `"billing exceeded"`, and `"billing account"`. Continues to match real 429-class messages (`"Billing account is not active"`, `"billing limit reached"`); stops matching unrelated billing-mentioning text.- Mistral / Gateway: bare `"billing"` likewise narrowed to `"billing limit"` / `"billing quota"`. Mistral additionally distinguishes `"out of credits"` vs `"insufficient credits"`.-- **Schema codegen cache bounded with LRU eviction**: `SchemaToModelFactory._schema_cache` now uses an `OrderedDict` capped at `_SCHEMA_CACHE_MAX_SIZE = 1024` entries. Prevents unbounded memory growth in long-running Temporal workers receiving many distinct schemas.- **Removed duplicate `ContentGenerationError` class**: the unused `pipelex.cogt.content_generation.exceptions.ContentGenerationError` was deleted (had zero external imports). Its subclasses re-parented: `NeitherUrlNorDataError` → `PipelexError`, `UnsafeSchemaError` → new `SecurityError`. The temporal `pipelex.temporal.exceptions.ContentGenerationError` is now the only class with that name.-- **Narrowed `WfPipeRun` exception handling**: `except Exception:` replaced with `ChildWorkflowError` (child-workflow path) and `ActivityError` (graph-assembly path). Real workflow bugs (TypeError, AttributeError, etc.) are no longer masked as pipe failures.--- **`PipeRouterTop` + `PipeRouterChild` merged into `TemporalPipeRouter`**: Single router that auto-detects context (top-level vs child workflow) and dispatches accordingly. Hub no longer needs `_pipe_router_top` — one `_pipe_router` field handles both modes.-- **`PipelexRunner.execute_pipeline()` uses `get_pipe_run()`**: Runner now delegates to the PipeRun layer instead of calling `get_pipe_router()` directly.-- **Tracing config structure**: `TracingConfig` now has `backend`, `ndjson` (with `traces_dir`), and `dynamodb` (with `table_name`, `region`) sub-configs. Tracing is enabled by default.-- **Event log factory replaces hardcoded NdjsonEventLog**: The 3 call sites (`pipeline_run_setup.py`, `wf_pipe_router.py`, `runner.py`) now use `make_event_log()` instead of directly instantiating `NdjsonEventLog`.-- **Tracing decoupled from Temporal**: Event log is created when `tracing_config.is_enabled`, regardless of whether Temporal is enabled.+- **Storage providers unified**: `StorageConfig` flattened to inheritance-based config so providers (`local`, `s3`) share a single shape across delivery and trace storage.- **`make rules` now generates both `CLAUDE.md` and `AGENTS.md` by default**: the `pipelex.kit_config.preferred_agent_target` setting has been renamed to `preferred_agent_targets` and is now a list. The default is `["claude", "agents"]`. Cursor remains exclusive (`["cursor"]`) and cannot be combined with single-file targets. Downstream projects overriding this setting must rename the key and wrap the value in a list.-- **All inference workers attach error category and user action**: Every worker across all providers now raises exceptions with an `InferenceErrorCategory` and actionable `user_action` hint.-- **CLI error output wired to `ErrorReport`**: Error handlers use `to_error_report()` for consistent, structured display.+- **Cold start trimmed**: heavy SDK imports (`boto3`, `huggingface_hub`, etc.) deferred behind `TYPE_CHECKING` or function-local guards across multiple plugins.- **Graph viewer updated to mthds-ui v0.3.4**: bumped from v0.3.0 — additional polish atop the resizable detail panel, escape-to-close, sticky header, prompt expand/collapse with copy button, concept refinement display.-- **README install instructions**: Replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section.--### Removed--- **`PipeRouterTop`** — replaced by `TemporalPipeRouter` + `TemporalPipeRun`.-- **`PipeRouterChild`** — merged into `TemporalPipeRouter`.-- **`PipelexHub.set_pipe_router_top()`** — no longer needed; single `set_pipe_router()` suffices.+- **Default models**: small-vision and creative defaults now point to `gemini-3.0-flash-preview`. Added `claude-4.6-sonnet` and Bedrock token-auth support.+- **`kajson` upgraded** from `0.3.1` to `0.4.2`.+- **README install instructions**: replaced step-by-step Claude Code setup with single copy-paste messages for Claude Code and Codex, added manual install section.### Fixed- **Anthropic structured generation no longer flattens all errors to `CONTENT`.** `AnthropicLLMWorker._gen_object` runs requests through `instructor`, which wraps SDK exceptions in `InstructorRetryException`. The previous handler categorized every wrapped error as `CONTENT`, so genuine `RateLimitError`, `APITimeoutError`, `APIConnectionError`, `PermissionDeniedError`, and `AuthenticationError` cases never reached the typed branches — they were reported as content failures (non-retryable) instead of `TRANSIENT` / `CAPACITY` / `CONFIGURATION`. The handler now unwraps `InstructorRetryException.failed_attempts[-1].exception` (with a `__cause__`/`RetryError.last_attempt` fallback) and routes recognized SDK exceptions through a shared categorization helper; truly unrecognized errors keep the `CONTENT` fallback.-- **ObjectAssignment deserialization on Temporal workers**: Removed `__init__` class registry check that blocked deserialization of `ObjectAssignment` before `library_crate` was loaded. Validation moved to `validate_before_execution()` method, following the existing codebase pattern.-- **Dynamic concept classes in Temporal activities**: `WfPipeRouter` now propagates dynamically registered concept classes from the per-workflow registry to the global registry, so child workflows and activities can access them.-- **Temporal nondeterminism in child workflow IDs**: Replaced `shortuuid.uuid()` with `workflow.uuid4()` in `TemporalPipeRouter` for deterministic child workflow ID generation. The old code caused TMPRL1100 errors on workflow replay.-- **Enum fields (`choices`) breaking Temporal serialization**: Concepts with `choices` fields generated `Enum` classes that couldn't be resolved during `model_rebuild` or deserialized across Temporal workflow boundaries. Fixed `_exec_and_extract_class` to include all generated types (models + enums) in the forward reference namespace.-- **Enum serialization in `ContentGeneratorChild`**: `model_dump()` now uses `mode="json"` to serialize enum values as plain strings, preventing type mismatch when validating across different model class instances.-- **`Anything[]` hydration in Temporal**: Working memory hydration failed for `Anything` concept lists because `AnythingContent` doesn't exist as a class. Hydration now gracefully falls back to embedded `__class__` metadata or `TextContent` for items without type information.- **Managed-deck detection no longer misclassifies user TOMLs.** `_is_managed_deck_filename` previously treated any non-`x_custom_*.toml` as kit-managed, so a user file like `my_overrides.toml` dropped into the deck dir would have been reported (and on `pipelex update` overwritten) as if it were a kit file. Now requires the numbered `^\d+_.*\.toml$` prefix that the kit actually ships.- **`PipeLLM` output structure prompt used the unresolved concept ref.** When `is_structure_prompt_enabled`, `get_output_structure_prompt` was called with `pipe_run_params.dynamic_output_concept_ref or output_stuff_spec.concept.concept_ref`. The first form can be a bare code (no domain), which broke downstream concept resolution. Now passes the already-resolved `output_stuff_spec.concept.concept_ref` directly.- **`PIPELEX_NO_DECK_NOTICE` suppression was too lax.** Any non-empty value silenced the deck-staleness notice — including `PIPELEX_NO_DECK_NOTICE=0`, which users might reasonably expect to _enable_ the notice. Now requires the documented `PIPELEX_NO_DECK_NOTICE=1`.- **Shipped `.kit_manifest.json` carried `kit_version: "0.25.0"`** for the 0.25.1 release, which would have triggered false stale-detection on fresh installs. Bumped to `0.25.1` and re-synced into `pipelex/kit/configs/`.+- **Stored-XSS guard**: HTML escaped in fallback `main_stuff.html`.+- **`GatewayExtractWorker` teardown**: guarded done-callbacks against cancelled tasks to stop noisy teardown errors.+- **URL checker**: 401/403 are now treated as OK for auth-walled URLs.+- **Pipeline duplicate guard**: registering the same pipeline twice raises explicitly instead of silently shadowing.### Documentation@@ -68,6 +85,8 @@### Security- **`schema_to_model` rejects `x-python-*` codegen extensions and restricts `__import__` to an allowlist.** `datamodel-code-generator` honors the `x-python-import` JSON Schema extension by emitting arbitrary `from <module> import <name>` statements with no sanitization. Combined with the prior gap that `_make_restricted_builtins()` did not block `__import__`, an attacker able to plant a crafted `object_class_schema` on a Temporal payload (where `ObjectAssignment.object_class_schema: dict[str, Any]` round-trips untouched) could cause arbitrary modules to be imported during `exec()` of generated code. Two-layer defense added: (1) `_reject_unsafe_schema_extensions` raises `UnsafeSchemaError` if any `x-python-*` key is present anywhere in the schema; (2) `__import__` in the exec namespace is now wrapped to allow only `pydantic`, `typing`, `typing_extensions`, `enum`, `datetime`, `decimal`, `uuid`, `__future__`, `collections`, and `re`.+- **Restricted exec builtins, path sanitization, and atomic fingerprint caching** in the schema-to-model pipeline.+- **CORS hardened on codec server error paths**; no leaked exception strings; prefix guards on storage URIs.## [v0.25.2] - 2026-04-30diff --git a/pipelex/cogt/content_generation/schema_to_model_factory.py b/pipelex/cogt/content_generation/schema_to_model_factory.pyindex 8cd3f33da..6a29bb9fe 100644--- a/pipelex/cogt/content_generation/schema_to_model_factory.py+++ b/pipelex/cogt/content_generation/schema_to_model_factory.py@@ -34,6 +34,7 @@ import reimport tempfileimport threadingfrom collections import OrderedDict+from enum import Enumfrom pathlib import Pathfrom typing import Any, ClassVar, cast@@ -72,6 +73,11 @@ class SchemaToModelFactory:_schema_cache: ClassVar[OrderedDict[str, type[BaseModel]]] = OrderedDict()_cache_lock: ClassVar[threading.Lock] = threading.Lock()+ _SOURCE_CACHE_MAX_SIZE: ClassVar[int] = 1024++ _source_cache: ClassVar[OrderedDict[str, dict[str, type[Any]]]] = OrderedDict()+ _source_cache_lock: ClassVar[threading.Lock] = threading.Lock()+@classmethoddef make_from_json_schema(cls, schema: dict[str, Any], class_name: str) -> type[BaseModel]:"""Reconstruct a BaseModel class from a JSON schema dict.@@ -216,8 +222,13 @@ class SchemaToModelFactory:return safe_builtins@classmethod- def _exec_and_extract_class(cls, source_code: str, class_name: str) -> type[BaseModel]:- """Execute source code and extract the named BaseModel class."""+ def _exec_source_to_types(cls, source_code: str) -> dict[str, type[Any]]:+ """Exec source through restricted builtins, return all top-level BaseModel + Enum+ subclasses with model_rebuild applied so forward references resolve.++ Internal primitive shared by both make_types_from_source (receiver path) and+ _exec_and_extract_class (sender path).+ """# TODO: replace exec() with programmatic construction via pydantic.create_model().# Eliminates the exec primitive entirely, removes the need for both security# layers, and lets us delete _make_restricted_builtins/_restricted_import.@@ -226,11 +237,52 @@ class SchemaToModelFactory:namespace: dict[str, Any] = {"__builtins__": cls._make_restricted_builtins()}exec(compile(source_code, "<schema_to_model_factory>", "exec"), namespace)- # Collect all BaseModel subclasses from the exec'd namespace- model_classes: dict[str, type[BaseModel]] = {- name: obj for name, obj in namespace.items() if isinstance(obj, type) and issubclass(obj, BaseModel) and obj is not BaseModel+ all_user_types: dict[str, type[Any]] = {+ name: obj+ for name, obj in namespace.items()+ if isinstance(obj, type)+ and not name.startswith("_")+ and (issubclass(obj, BaseModel) or issubclass(obj, Enum))+ and obj is not BaseModel+ and obj is not Enum}+ # datamodel-code-generator uses `from __future__ import annotations` which turns+ # type annotations into strings. Rebuild every BaseModel so forward refs (including+ # references to generated Enum classes for choices fields) resolve against the+ # full type namespace.+ for candidate in all_user_types.values():+ if issubclass(candidate, BaseModel):+ candidate.model_rebuild(_types_namespace=all_user_types)++ return all_user_types++ @classmethod+ def make_types_from_source(cls, source_code: str) -> dict[str, type[Any]]:+ """Receiver-side entry: exec generated source and return all top-level user-defined+ types (BaseModel + Enum subclasses) keyed by class name.++ Cached by sha256(source_code) so repeated payloads carrying the same dynamic+ class don't re-exec. Returns a shallow copy so callers can mutate freely.+ """+ cache_key = hashlib.sha256(source_code.encode()).hexdigest()+ with cls._source_cache_lock:+ if cache_key in cls._source_cache:+ cls._source_cache.move_to_end(cache_key)+ return dict(cls._source_cache[cache_key])++ types_dict = cls._exec_source_to_types(source_code)+ if len(cls._source_cache) >= cls._SOURCE_CACHE_MAX_SIZE:+ cls._source_cache.popitem(last=False)+ cls._source_cache[cache_key] = types_dict+ return dict(types_dict)++ @classmethod+ def _exec_and_extract_class(cls, source_code: str, class_name: str) -> type[BaseModel]:+ """Execute source code and extract the named BaseModel class."""+ all_user_types = cls.make_types_from_source(source_code)+ model_classes: dict[str, type[BaseModel]] = {name: obj for name, obj in all_user_types.items() if issubclass(obj, BaseModel)}+# datamodel-code-generator normalizes schema titles to PascalCase class names# (e.g. "dynamic_concept_test__Greeting" → "DynamicConceptTestGreeting"),# so try both the original name and the normalized version.@@ -243,13 +295,4 @@ class SchemaToModelFactory:msg = f"Class '{class_name}' not found in generated source. Available BaseModel classes: {available}"raise ValueError(msg)- # datamodel-code-generator uses `from __future__ import annotations` which turns- # type annotations into strings. For nested models, we need to rebuild so Pydantic- # resolves the forward references against all classes in the generated namespace.- # Include ALL user-defined types (models + enums) so forward references to- # generated Enum classes (e.g. choices fields) can be resolved.- all_generated_types: dict[str, Any] = {name: obj for name, obj in namespace.items() if isinstance(obj, type) and not name.startswith("_")}- for model_cls in model_classes.values():- model_cls.model_rebuild(_types_namespace=all_generated_types)-return extracted_classdiff --git a/pipelex/temporal/temporal_data_converter.py b/pipelex/temporal/temporal_data_converter.pyindex d6d58dc70..bd840b927 100644--- a/pipelex/temporal/temporal_data_converter.py+++ b/pipelex/temporal/temporal_data_converter.py@@ -28,6 +28,7 @@ For examples, see the tests in `test_top_crafter.py`:from typing import Any, castfrom kajson import kajson+from kajson.class_registry import ClassRegistryfrom pydantic import BaseModelfrom temporalio.api.common.v1 import Payloadfrom temporalio.converter import (@@ -41,6 +42,7 @@ from temporalio.converter import (from typing_extensions import overridefrom pipelex import log+from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactoryfrom pipelex.hub import get_class_registryfrom pipelex.temporal.exceptions import TemporalFlowError@@ -89,11 +91,22 @@ class BaseModelPayloadConverter(JSONPlainPayloadConverter):log.verbose(f"unijson_deserialize_payload — data: {data}")source_bytes = payload.metadata.get("kajson_class_source")class_source_code = source_bytes.decode() if source_bytes else None- pydantic_gizmo = kajson.loads(- data,- class_registry=get_class_registry(),- class_source_code=class_source_code,- )++ global_registry = get_class_registry()+ if class_source_code is not None:+ # Build a per-call scoped registry overlaying the global one with the+ # source-derived dynamic classes (BaseModel + Enum). Per-call so dynamic+ # classes never persist across payloads — different schemas may reuse the+ # same class name without collision.+ source_types = SchemaToModelFactory.make_types_from_source(class_source_code)+ scoped_registry = ClassRegistry()+ scoped_registry.register_classes_dict(global_registry.get_classes_dict())+ for type_name, type_obj in source_types.items():+ scoped_registry.register_class(type_obj, name=type_name, should_warn_if_already_registered=False)+ pydantic_gizmo = kajson.loads(data, class_registry=scoped_registry)+ else:+ pydantic_gizmo = kajson.loads(data, class_registry=global_registry)+if class_source_code is not None:if isinstance(pydantic_gizmo, BaseModel):self._restore_class_source(pydantic_gizmo, class_source_code)diff --git a/pipelex/temporal/tprl_pipe/submitter_hydration.py b/pipelex/temporal/tprl_pipe/submitter_hydration.pyindex 4c5015e32..7b4e43ba7 100644--- a/pipelex/temporal/tprl_pipe/submitter_hydration.py+++ b/pipelex/temporal/tprl_pipe/submitter_hydration.py@@ -23,7 +23,6 @@ from typing import TYPE_CHECKINGfrom kajson.class_registry import ClassRegistryfrom kajson.kajson_manager import KajsonManager-from pipelex import logfrom pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.hub import get_current_library, get_library_manager, set_current_library, teardown_current_libraryfrom pipelex.temporal.tprl_pipe.hydration import hydrate_working_memory@@ -66,10 +65,7 @@ def rehydrate_pipe_output_with_crate(global_registry = KajsonManager.get_class_registry()scoped_registry = ClassRegistry()- if isinstance(global_registry, ClassRegistry):- scoped_registry.register_classes_dict(dict(global_registry.root))- else:- log.warning(f"Global registry is {type(global_registry).__name__}, not ClassRegistry — cannot pre-seed rehydration registry")+ scoped_registry.register_classes_dict(global_registry.get_classes_dict())rehydration_library.set_class_registry(scoped_registry)set_current_library(library_id=rehydration_library_id)diff --git a/pipelex/temporal/tprl_pipe/wf_pipe_router.py b/pipelex/temporal/tprl_pipe/wf_pipe_router.pyindex a728e3435..274d18b2e 100644--- a/pipelex/temporal/tprl_pipe/wf_pipe_router.py+++ b/pipelex/temporal/tprl_pipe/wf_pipe_router.py@@ -52,10 +52,7 @@ class WfPipeRouter(WorkflowClass[PipeJob, PipeOutput]):# 1. Create per-workflow ClassRegistry pre-seeded from globalglobal_registry = KajsonManager.get_class_registry()workflow_registry = ClassRegistry()- if isinstance(global_registry, ClassRegistry):- workflow_registry.register_classes_dict(dict(global_registry.root))- else:- workflow_log.warning("Global registry is not a ClassRegistry, cannot pre-seed workflow registry")+ workflow_registry.register_classes_dict(global_registry.get_classes_dict())# 2. Open library and attach registry to itlibrary_manager = get_library_manager()diff --git a/pyproject.toml b/pyproject.tomlindex 1d90e6acd..6fe687d86 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -1,6 +1,6 @@[project]name = "pipelex"-version = "0.25.2"+version = "0.26.0"description = "Execute composable AI methods declared in the MTHDS open standard"authors = [{ name = "Evotis S.A.S.", email = "[email protected]" }]maintainers = [{ name = "Pipelex staff", email = "[email protected]" }]@@ -24,10 +24,10 @@ dependencies = ["datamodel-code-generator>=0.55.0","filetype>=1.2.0","httpx>=0.23.0,<1.0.0",- "instructor>=1.8.3,!=1.11.*,!=1.12.*", # 1.11.x caused typing errors with mypy+ "instructor>=1.8.3,!=1.11.*,!=1.12.*", # 1.11.x caused typing errors with mypy"jinja2>=3.1.4","json2html>=1.3.0",- "kajson @ git+https://github.com/Pipelex/kajson.git@fix/enum-source-registry",+ "kajson==0.5.0","markdown>=3.6","mthds>=0.3.0","networkx>=3.4.2",@@ -44,7 +44,7 @@ dependencies = ["pydantic>=2.10.6,<3.0.0","python-dotenv>=1.0.1","PyYAML>=6.0.2",- "reportlab>=4.0,<5", # for generating synthetic PDFs+ "reportlab>=4.0,<5", # for generating synthetic PDFs"rich>=13.8.1","semantic-version>=2.10.0","shortuuid>=1.0.13",diff --git a/tests/integration/pipelex/fixtures/pipe_job_helpers.py b/tests/integration/pipelex/fixtures/pipe_job_helpers.pyindex fb0b10f3b..48498e9d4 100644--- a/tests/integration/pipelex/fixtures/pipe_job_helpers.py+++ b/tests/integration/pipelex/fixtures/pipe_job_helpers.py@@ -67,8 +67,7 @@ def pipe_job_from_library(# this scoped registry instead of the global, keeping the global clean.global_registry = KajsonManager.get_class_registry()scoped_registry = ClassRegistry()- if isinstance(global_registry, ClassRegistry):- scoped_registry.register_classes_dict(dict(global_registry.root))+ scoped_registry.register_classes_dict(global_registry.get_classes_dict())library.set_class_registry(scoped_registry)set_current_library(library_id=library_id)diff --git a/tests/integration/pipelex/temporal/data_converter/test_data_conv_enum_roundtrip.py b/tests/integration/pipelex/temporal/data_converter/test_data_conv_enum_roundtrip.pynew file mode 100644index 000000000..3ad2c80af--- /dev/null+++ b/tests/integration/pipelex/temporal/data_converter/test_data_conv_enum_roundtrip.py@@ -0,0 +1,57 @@+from enum import Enum+from typing import cast++import pytest+from pydantic import BaseModel++from pipelex import pretty_print+from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory+from pipelex.temporal.temporal_data_converter import BaseModelPayloadConverter+from pipelex.types import StrEnum+++class PetSpecies(StrEnum):+ DOG = "dog"+ CAT = "cat"+++class Pet(BaseModel):+ name: str+ species: PetSpecies+++class TestEnumRoundTrip:+ def test_dynamic_class_with_enum_field_round_trips(+ self,+ payload_converter: BaseModelPayloadConverter,+ ):+ """A class with an Enum field generated via SchemaToModelFactory must survive a+ full payload round-trip. Exercises the receiver-side exec path that registers+ Enum subclasses in the per-call scoped ClassRegistry — without it the+ deserializer cannot resolve the dynamic enum class and the round-trip fails.+ """+ schema = Pet.model_json_schema()+ dynamic_pet_cls = SchemaToModelFactory.make_from_json_schema(schema, "Pet")+ instance = dynamic_pet_cls(name="Rex", species=PetSpecies.DOG)+ pretty_print(instance, title="instance")++ payload = payload_converter.to_payload(instance)+ pretty_print(payload, title="payload")+ assert payload is not None+ assert payload.metadata.get("kajson_class_source")++ restored = cast("BaseModel", payload_converter.from_payload(payload, type_hint=BaseModel))+ pretty_print(restored, title="restored")++ restored_class: type[BaseModel] = type(restored)+ assert restored_class.__name__ == "Pet"+ assert restored.name == "Rex" # type: ignore[attr-defined]+ # datamodel-code-generator emits `class PetSpecies(Enum)` (not StrEnum), so the+ # dynamic enum is a distinct class from the static PetSpecies above. Assert on+ # class name + value instead of identity equality.+ species_value: Enum = restored.species # type: ignore[attr-defined]+ assert isinstance(species_value, Enum)+ assert type(species_value).__name__ == "PetSpecies"+ assert species_value.value == "dog"+ assert getattr(restored_class, "__kajson_class_source__", None)diff --git a/uv.lock b/uv.lockindex 4539a3f18..b02cffc67 100644--- a/uv.lock+++ b/uv.lock@@ -1938,11 +1938,15 @@ wheels = [[[package]]name = "kajson"-version = "0.4.2"-source = { git = "https://github.com/Pipelex/kajson.git?rev=fix%2Fenum-source-registry#4e80ebf729e22f4f3a6a4fac846a024e38fc2fd0" }+version = "0.5.0"+source = { registry = "https://pypi.org/simple" }dependencies = [{ name = "pydantic" },]+sdist = { url = "https://files.pythonhosted.org/packages/91/9c/28a44cc51f328fe1076ea2114b6b7791ef49c54c24a07f11f86f83438da0/kajson-0.5.0.tar.gz", hash = "sha256:029b6a1b60ea28ec95b6f614656d1e48600e27c1f27a39c6e5681cd721854f64", size = 21969, upload-time = "2026-05-04T12:20:12.734Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/2e/f4/6460fb07535ef71f87416e1d603f7eaaa9d2970161bec3e37fe08af74b45/kajson-0.5.0-py3-none-any.whl", hash = "sha256:53334fd7122e470ee560255eb5a8390b5ad60ea99bc0d6b75122b323379b553a", size = 28398, upload-time = "2026-05-04T12:20:11.202Z" },+][[package]]name = "latex2mathml"@@ -3626,7 +3630,7 @@ wheels = [[[package]]name = "pipelex"-version = "0.25.2"+version = "0.26.0"source = { editable = "." }dependencies = [{ name = "aiofiles" },@@ -3774,7 +3778,7 @@ requires-dist = [{ name = "instructor", extras = ["google-genai"], marker = "extra == 'google-genai'" },{ name = "jinja2", specifier = ">=3.1.4" },{ name = "json2html", specifier = ">=1.3.0" },- { name = "kajson", git = "https://github.com/Pipelex/kajson.git?rev=fix%2Fenum-source-registry" },+ { name = "kajson", specifier = "==0.5.0" },{ name = "linkup-sdk", marker = "extra == 'linkup'", specifier = ">=0.12.0" },{ name = "lxml", marker = "extra == 'docling'", specifier = ">=6.1.0" },{ name = "markdown", specifier = ">=3.6" },