Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Features and Docs and tests to generate structure from Pipeline's toml, inc. inline in Concept def

Louis Choquel00e0477e67fa
diff --git a/.pipelex/create/design_pipelines.md b/.pipelex/create/design_pipelines.md
deleted file mode 100644
index 2a3fcfb80..000000000
--- a/.pipelex/create/design_pipelines.md
+++ /dev/null
@@ -1,270 +0,0 @@
-## Designing Pipelex pipelines
-
-This section explains how to design pipelines for Pipelex.
-
-### Workflow for a designer agent
-
-- Plan the pipeline in natural language (goal, inputs, steps, outputs).
-
-- Translate the plan into a `PipelineBlueprint` object graph: domain, concepts, pipes.
-
-- Define controllers first (`PipeSequence`, `PipeParallel`, `PipeCondition`), then operators (`PipeLLM`, `PipeOcr`, etc.).
-
-- For each new pipe, detail its `type` and `definition`.
-
-- Ensure each pipe’s `inputs` shows all required variables, including those used by nested steps or conditional branches. `inputs` is a dict mapping variable names to concept codes.
-
-- Choose appropriate native or structured output concepts.
-
-### Root blueprint object: PipelineBlueprint
-
-`PipelineBlueprint` fields:
-
-- `domain: str` — required. Single word or snake_case code for the domain.
-
-- `definition: Optional[str]` — optional textual description of the domain.
-
-- `system_prompt: Optional[str]` — default system prompt applied by LLM pipes in this domain if they don’t provide one.
-
-- `system_prompt_to_structure: Optional[str]` — domain-level instructions to structure outputs.
-
-- `prompt_template_to_structure: Optional[str]` — domain-level instructions to structure prompts.
-
-- `concept: Dict[str, Union[str, ConceptBlueprint]]` — concept section. Keys are concept names (PascalCase). Values are either:
-
- - a string (quick form): the concept definition; or
-
- - a `ConceptBlueprint` object (full form).
-
-- `pipe: Dict[str, Dict[str, Any]]` — pipes section. Keys are pipe codes (snake_case). Values are per-pipe details dicts that will be validated into concrete `PipeBlueprint` subclasses based on their `type`.
-
-### Concepts
-
-- Conceiving and naming concepts:
- - Use PascalCase for concept names.
-
- - DO NOT use plurals in concept names (use `Item` not `Items`).
-
- - DO NOT use adjectives in concept names (use `Text`, not `LargeText`).
-
- - When a concept or concept reference is unqualified (no `domain.` prefix), it will be prefixed with the current blueprint `domain` during validation/expansion, unless its name corresponds to one of the native concepts.
-
- - Do not redefine native concepts (use them directly): `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `LLMPrompt`.
-
- - Textual concepts refining `native.Text` are acceptable too, and you don't need to specify their structure: by default it'll be text.
-
- - Reuse existing concepts whenever possible. For instance if you take a `Poem` and make it dark, the result is a `Poem` too: you MUST NOT create a new `DarkPoem` concept in this kind of situation.
-
- - Concepts from the base library are also provided and must be used whenever possible:
-
- - `documents.TextAndImagesContent` = "A content that comprises text and images where the text can include local links to the images"
-
- - `images.VisualDescription` = "Visual description of something"
-
- - `images.ImgGenPrompt`:
- - definition = "Prompt to generate an image"
- - refines = "Text"
-
- - `images.Photo`:
- - definition = "Photo"
- - structure = "ImageContent"
- - refines = "Image"
-
- - DO NOT create a concept qualified with circumstances like `InputPhoto`: it's just a `Photo`, it so happens that we're using it as input, OK, but that will be convened by the rest of the structure.
-
-- Quick form (string):
-
- - `concept["ConceptName"] = "Definition of the concept"`
-
- - If no `structure` is specified and the name isn’t a known structure class, it implicitly refines `native.Text`.
-
-- Full form (`ConceptBlueprint` from `pipelex/core/concept_factory.py`):
-
- - `definition: str` — required.
-
- - `structure: Optional[str]` — name of a registered `StructuredContent` subclass (typically defined in `pipelex_libraries/pipelines/<domain>.py`). If provided, it must exist in the class registry.
-
- - `refines: Union[str, List[str]]` — concept codes or names it refines. May be a single string or a list. Unqualified names are prefixed with the current `domain` automatically.
-
- - `domain: Optional[str]` — not required when provided via the library loader.
-
-### Pipes: overall rules
-
-Each entry under `pipe` is a dict that will be converted into a typed `PipeBlueprint` (see `pipelex/core/pipe_blueprint.py` for the base and factories in `pipe_controllers/` and `pipe_operators/`). Use the new format:
-
-- `type: str` — the pipe class name, e.g., `PipeLLM`, `PipeOcr`, `PipeSequence`, `PipeParallel`, `PipeCondition`, `PipeFunc`, `PipeImgGen`.
-
-- `definition: str` — human-readable description of the pipe’s responsibility.
-
-- `inputs: Optional[Dict[str, str]]` — map of variable name -> concept code or name. Unqualified names are prefixed with `domain`.
-
-- `output: str` — target concept code or name. Unqualified names are prefixed with `domain`.
-
-Important:
-
-- For controller pipes (`PipeSequence`, `PipeParallel`, `PipeCondition`), the parent pipe’s `inputs` must include all variables required by its sub-pipes or conditional branches that come from the initial working memory (i.e., not produced by previous steps).
-
-- Multiplicity: some pipes and sub-steps can produce multiple outputs (see below). Plan result names accordingly.
-
-### Pipe-specific blueprint schemas
-
-Below are the additional fields expected per type, on top of the base fields above (`type`, `definition`, `inputs`, `output`). All fields are validated by their blueprint classes.
-
-#### PipeLLM (operators/pipe_llm_factory.py)
-
-- Prompt sources (choose at most one per group):
-
- - System prompt: one of `system_prompt`, `system_prompt_name`, `system_prompt_template`, `system_prompt_template_name`.
-
- - User prompt: one of `prompt`, `prompt_name`, `prompt_template`, `template_name`.
-
-- LLM selection:
-
- - `llm: Optional[LLMSettingOrPresetId]` — model for text.
-
- - `llm_to_structure: Optional[LLMSettingOrPresetId]` — model for object structuring.
-
-- Structuring:
-
- - `structuring_method: Optional[StructuringMethod]`
-
- - `prompt_template_to_structure: Optional[str]`
-
- - `system_prompt_to_structure: Optional[str]`
-
-- Multiplicity:
-
- - `nb_output: Optional[int]` — exact number of outputs; mutually exclusive with `multiple_output`.
-
- - `multiple_output: Optional[bool]` — allow variable number of outputs.
-
-Behavioral notes:
-
-- If no system prompt is provided, the domain’s `system_prompt` (from the root blueprint) is used if available.
-
-- Inputs that are `Image` concepts are automatically exposed as user images to the LLM; other inputs become text blocks.
-
-- Prompt template insertion style for the agent:
-
- - Use block insertion (equivalent to `@name`) for long, multi-line stuff.
-
- - Use inline insertion (equivalent to `$name`) for short tokens embedded in sentences.
-
- - Do not synthesize literal markers; provide structured prompt fields and let the runtime render appropriately.
-
-#### PipeOcr (operators/pipe_ocr_factory.py)
-
-- Required input variable name: `ocr_input`.
-
-- The `ocr_input` concept must be `Image` or `PDF`.
-
-- Additional options:
-
- - `ocr_platform: Optional[OcrPlatform]` — defaults to Mistral.
-
- - `page_images: bool` — include detected images.
-
- - `page_image_captions: bool` — caption images.
-
- - `page_views: bool` — include rendered page views.
-
- - `page_views_dpi: Optional[int]` — DPI for page views; defaults to config.
-
-- Output concept should be `Page`. The operational result is a list of `Page` items.
-
-#### PipeSequence (controllers/pipe_sequence_factory.py)
-
-- `steps: List[SubPipeBlueprint]`
-
-- `SubPipeBlueprint` fields (controllers/sub_pipe_factory.py):
-
- - `pipe: str` — referenced pipe code to execute.
-
- - `result: Optional[str]` — name to assign to this step’s output in working memory.
-
- - Multiplicity (choose at most one): `nb_output: Optional[int]`, `multiple_output: Optional[bool]`.
-
- - Batch (both or none): `batch_over: Union[bool, str]` and `batch_as: Optional[str]`.
-
-Rules:
-
-- If `batch_over` is provided, `batch_as` is required; if `batch_as` is provided, `batch_over` is required.
-
-- When batching, the step runs in parallel over each element of the specified list, and the `result` becomes a list.
-
-#### PipeParallel (controllers/pipe_parallel_factory.py)
-
-- `parallels: List[SubPipeBlueprint]` — each must specify `result`.
-
-- `add_each_output: bool = True` — whether to add each parallel result separately to memory.
-
-- `combined_output: Optional[str]` — optional combined output concept code; if provided without a domain prefix, it’s prefixed automatically.
-
-Rules:
-
-- At least one of `add_each_output` or `combined_output` must be set appropriately (cannot disable both behaviors).
-
-#### PipeCondition (controllers/pipe_condition_factory.py)
-
-- Choose exactly one of:
-
- - `expression: Optional[str]` — direct expression.
-
- - `expression_template: Optional[str]` — Jinja2 template expression.
-
-- `pipe_map: Dict[str, str]` — map of evaluated expression values to pipe codes to execute.
-
-- `default_pipe_code: Optional[str]` — optional fallback pipe code if no key matches.
-
-- `add_alias_from_expression_to: Optional[str]` — if set, store the evaluated expression under this variable name in working memory.
-
-### Input and output concepts resolution
-
-- Unqualified concept names in `inputs` and `output` are automatically prefixed with the current `domain` during blueprint validation (see `PipeBlueprint.add_domain_prefix`).
-
-- For `PipeParallel.combined_output`, the same domain-prefixing rule applies.
-
-### Checklist for the agent
-
-- Define `domain` and optional domain-level prompts (`system_prompt`, `system_prompt_to_structure`, `prompt_template_to_structure`).
-
-- Create `concept` entries using either quick string form or full `ConceptBlueprint` with `definition`, optional `structure`, and `refines`.
-
-- For each pipe under `pipe`:
-
- - Provide `type`, `definition`, `inputs`, `output`.
-
- - Add type-specific fields (see sections above).
-
- - Ensure controller pipes include all upstream inputs needed by nested steps/branches.
-
- - Set multiplicity (`nb_output`/`multiple_output`) and batch settings where needed.
-
- - For OCR, use `ocr_input` with `Image` or `PDF`, and set `output` to `Page`.
-
-- Validate mentally that every referenced sub-pipe exists and every input it requires is either provided by the parent `inputs` or produced by a previous step.
-
-### Minimal skeleton (object form)
-
-The agent should generate a structure similar to the following (illustrative, not exhaustive):
-
-```json
-{
- "domain": "my_domain",
- "definition": "What this domain covers",
- "system_prompt": "Global system prompt for LLM pipes (optional)",
- "concept": {
- "UserQuery": "Natural language question from a user",
- "Answer": { "definition": "Answer text" }
- },
- "pipe": {
- "answer_question": {
- "type": "PipeLLM",
- "definition": "Produce an answer from a question and context",
- "inputs": { "question": "UserQuery", "context": "Text" },
- "output": "Answer",
- "prompt_template": "Analyze the question and context, then answer concisely.\n\n@question\n\n@context"
- }
- }
-}
-```
diff --git a/.pipelex/create/structures.md b/.pipelex/create/structures.md
new file mode 100644
index 000000000..bd7df9127
--- /dev/null
+++ b/.pipelex/create/structures.md
@@ -0,0 +1,396 @@
+# Defining Structures for Pipelex Pipelines
+
+This guide explains how to define schemas and enums for structured data generation in multi-step workflows. This step occurs between drafting pipelines (see `draft_pipelines.md`) and building them (see `build_pipelines.md`).
+
+## When to Define Structures
+
+After drafting your pipeline but before building it, review your draft for:
+
+1. **Complex data that needs validation**: When your pipeline generates structured information that must follow specific formats
+2. **Multi-field outputs**: When a single "result" actually contains multiple related pieces of information
+3. **Constrained values**: When fields can only have specific allowed values
+4. **Reusable data types**: When the same structure appears in multiple places
+
+## Structure Definition Workflow
+
+### Step 1: Identify Structure Needs from Draft
+
+Review your pipeline draft and look for:
+
+```markdown
+# Example draft excerpt
+LLM[analyze]: (document) → analysis_report
+LLM[extract]: (page) → contact_info
+LLM[classify]: (query) → support_category
+```
+
+Ask yourself:
+- What specific fields does `analysis_report` contain?
+- What format should `contact_info` follow?
+- What are the valid values for `support_category`?
+
+### Step 2: Design Schema Definitions
+
+Create a TOML file defining your structures using the syntax from `structured_output_generator.py`.
+
+## Structure Definition Syntax
+
+### Basic Structure Definition
+
+```toml
+[structure.ModelName]
+definition = "Clear description of what this represents"
+
+[structure.ModelName.fields]
+field_name = "Field description"
+another_field = { type = "text", definition = "More detailed field", required = true }
+```
+
+### Field Types
+
+| Type | Description | Example |
+|------|-------------|---------|
+| `text` | String content | `name = "Person's full name"` |
+| `number` | Floating point | `score = { type = "number", definition = "Confidence score" }` |
+| `integer` | Whole numbers | `count = { type = "integer", definition = "Number of items" }` |
+| `boolean` | True/false | `active = { type = "boolean", definition = "Is active" }` |
+| `list` | Array of items | `tags = { type = "list", item_type = "text", definition = "List of tags" }` |
+| `dict` | Key-value pairs | `metadata = { type = "dict", key_type = "text", value_type = "text" }` |
+
+### Field Configuration Options
+
+```toml
+# Required field (must be provided)
+title = { type = "text", definition = "Document title", required = true }
+
+# Optional field with default value
+priority = { type = "integer", definition = "Priority level", default = 1 }
+
+# List with specific item type
+categories = { type = "list", item_type = "text", definition = "Document categories" }
+```
+
+## Enum Definition Syntax
+
+### Simple Enum (List of Values)
+
+```toml
+[enum.Priority]
+definition = "Task priority levels"
+values = ["low", "medium", "high", "urgent"]
+```
+
+### Descriptive Enum (Key-Value Pairs)
+
+```toml
+[enum.DocumentType]
+definition = "Types of documents we can process"
+
+[enum.DocumentType.values]
+technical = "Technical documentation and manuals"
+legal = "Legal contracts and agreements"
+financial = "Financial reports and statements"
+marketing = "Marketing materials and content"
+```
+
+### Using Enums in Structures
+
+```toml
+# Reference defined enum
+status = { type = "Priority", definition = "Current priority level" }
+
+# Inline choices for simple cases
+size = { choices = ["S", "M", "L", "XL"], definition = "Available sizes" }
+```
+
+## Complete Example: Document Analysis Pipeline
+
+### From Draft to Structures
+
+**Original Draft Step:**
+```markdown
+### Step 2: Extract structured information from each page
+LLM[extract]: (page) → page_info
+
+### Step 3: Classify document type and priority
+LLM[classify]: (document_content) → document_metadata
+
+### Step 4: Generate comprehensive analysis
+LLM[analyze]: (page_info_list, document_metadata) → final_analysis
+```
+
+**Structure Definition (document_analysis_structures.toml):**
+
+```toml
+# Enums for constrained values
+[enum.DocumentType]
+definition = "Types of documents we can process"
+values = ["technical", "legal", "financial", "marketing", "general"]
+
+[enum.Priority]
+definition = "Document processing priority"
+values = ["low", "medium", "high", "urgent"]
+
+[enum.ConfidenceLevel]
+definition = "AI confidence in extraction"
+
+[enum.ConfidenceLevel.values]
+low = "Low confidence - manual review recommended"
+medium = "Medium confidence - spot check suggested"
+high = "High confidence - likely accurate"
+
+# Structures for structured data
+[structure.PageInfo]
+definition = "Structured information extracted from a document page"
+
+[structure.PageInfo.fields]
+page_number = { type = "integer", definition = "Page number in document", required = true }
+main_content = "Primary text content of the page"
+key_points = { type = "list", item_type = "text", definition = "Important points found on page" }
+entities = { type = "list", item_type = "text", definition = "Named entities (people, organizations, etc.)" }
+confidence = { type = "ConfidenceLevel", definition = "Confidence in extraction quality" }
+
+[structure.DocumentMetadata]
+definition = "High-level document classification and properties"
+
+[structure.DocumentMetadata.fields]
+document_type = { type = "DocumentType", definition = "Classified document type", required = true }
+priority = { type = "Priority", definition = "Processing priority", required = true }
+title = "Inferred document title"
+summary = "Brief document summary"
+page_count = { type = "integer", definition = "Total number of pages" }
+language = { choices = ["en", "fr", "es", "de"], definition = "Document language" }
+
+[structure.DocumentAnalysis]
+definition = "Comprehensive analysis combining all extracted information"
+
+[structure.DocumentAnalysis.fields]
+document_metadata = { type = "DocumentMetadata", definition = "Document classification info", required = true }
+page_analyses = { type = "list", item_type = "PageInfo", definition = "Analysis for each page", required = true }
+overall_insights = { type = "list", item_type = "text", definition = "Key insights across entire document" }
+recommendations = { type = "list", item_type = "text", definition = "Recommended actions based on analysis" }
+confidence_score = { type = "number", definition = "Overall confidence in analysis (0.0-1.0)" }
+```
+
+## Best Practices
+
+### 1. Semantic Naming
+- Use **PascalCase** for schema and enum names: `DocumentAnalysis`, `Priority`
+- Use **snake_case** for field names: `page_number`, `confidence_score`
+- Avoid plurals in schema names: `PageInfo` not `PageInfos`
+- Avoid circumstantial names: `Analysis` not `FinalAnalysis`
+
+### 2. Clear Definitions
+- Every schema and enum should have a `definition`
+- Field descriptions should explain what the field contains, not how it's used
+- Use complete sentences for definitions
+
+### 3. Appropriate Constraints
+- Mark fields as `required = true` only when they're always expected
+- Use enums for fields with limited, known values
+- Use inline `choices` for simple, one-off constraints
+- Set reasonable defaults where applicable
+
+### 4. Logical Grouping
+- Group related fields into the same schema
+- Don't create overly granular schemas for simple data
+- Consider if a field should be separate or part of a larger structure
+
+### 5. Reusability
+- Design schemas to be reusable across different pipes
+- Create enums for values that appear in multiple contexts
+- Avoid pipeline-specific naming in reusable structures
+
+## Integration with Pipeline Building
+
+### Updating Concept Definitions
+
+After defining structures, update your pipeline blueprint concepts to reference the structure definitions:
+
+**Simple approach (structure name matches concept name):**
+```json
+"concept": {
+ "PageInfo": "Information extracted from a page",
+ "DocumentMetadata": "Document classification data",
+ "DocumentAnalysis": "Complete document analysis"
+}
+```
+
+**Structure reference approaches:**
+```json
+"concept": {
+ "PageInfo": {
+ "definition": "Structured information extracted from a document page",
+ "structure": "PageInfo"
+ },
+ "DocumentMetadata": {
+ "definition": "High-level document classification and properties",
+ "structure": {
+ "document_type": {
+ "type": "DocumentType",
+ "definition": "Classified document type",
+ "required": true
+ },
+ "priority": {
+ "type": "Priority",
+ "definition": "Processing priority",
+ "required": true
+ },
+ "title": "Inferred document title",
+ "summary": "Brief document summary",
+ "page_count": {
+ "type": "integer",
+ "definition": "Total number of pages"
+ },
+ "language": {
+ "choices": ["en", "fr", "es", "de"],
+ "definition": "Document language"
+ }
+ }
+ },
+ "DocumentAnalysis": {
+ "definition": "Comprehensive analysis combining all extracted information",
+ "structure": "DocumentAnalysis"
+ }
+}
+```
+
+**Structure attribute options:**
+- **String reference**: `"structure": "ClassName"` - References a predefined structure class
+- **Inline definition**: `"structure": { ... }` - Defines fields directly within the concept
+- **Auto-detection**: Omit `structure` entirely - System looks for StructuredContent subclass with matching concept name or falls back on `Text` (i.e. not structured)
+
+### When to Use Each Approach
+
+**Use string reference when:**
+- Structure is reusable across multiple concepts or pipelines
+- Structure is complex and benefits from being defined in a separate TOML file
+- You want to maintain separation between concept definitions and structure details
+
+**Use inline definition when:**
+- Structure is specific to this one concept
+- Structure is simple and doesn't warrant a separate definition
+- You want to keep concept and structure together for clarity
+- Rapid prototyping where you don't want to manage separate files
+
+**Use auto-detection when:**
+- Concept name exactly matches an existing StructuredContent class
+- Working with simple, standard structures that are already registered
+
+### Generating Python Classes
+
+**From TOML files:**
+```python
+from pipelex.create.structured_output_generator import generate_structured_outputs_from_toml_file
+
+# Generate Python classes from TOML definitions
+generate_structured_outputs_from_toml_file(
+ "document_analysis_structures.toml",
+ "document_analysis.py"
+)
+```
+
+**From inline definitions (for concept integration):**
+```python
+from pipelex.create.structured_output_generator import generate_structured_output_from_inline_definition
+
+# Generate from inline field definitions (as used in concept.structure)
+fields_def = {
+ "document_type": {
+ "type": "DocumentType",
+ "definition": "Classified document type",
+ "required": True
+ },
+ "priority": {
+ "type": "Priority",
+ "definition": "Processing priority",
+ "required": True
+ },
+ "title": "Inferred document title",
+ "page_count": {
+ "type": "integer",
+ "definition": "Total number of pages"
+ },
+ "language": {
+ "choices": ["en", "fr", "es", "de"],
+ "definition": "Document language"
+ }
+}
+
+enums = {
+ "DocumentType": {
+ "definition": "Types of documents we can process",
+ "values": ["technical", "legal", "financial", "marketing", "general"]
+ },
+ "Priority": {
+ "definition": "Document processing priority",
+ "values": ["low", "medium", "high", "urgent"]
+ }
+}
+
+python_code = generate_structured_output_from_inline_definition(
+ "DocumentMetadata",
+ fields_def,
+ enums
+)
+```
+
+## Validation Checklist
+
+Before finalizing your structure definitions:
+
+- [ ] All structure names use PascalCase and are singular
+- [ ] All field names use snake_case
+- [ ] Every structure and enum has a clear `definition`
+- [ ] Required fields are marked appropriately
+- [ ] Enums are used for constrained values with known options
+- [ ] Field types match the expected data format
+- [ ] Structures group related fields logically
+- [ ] Structure names are semantic (describe what they are, not how they're used)
+- [ ] No native concepts (Text, Image, PDF) are redefined
+- [ ] List item types are specified correctly
+- [ ] Dictionary key and value types are appropriate
+
+## Common Patterns
+
+### Contact Information
+```toml
+[structure.ContactInfo]
+definition = "Contact information extracted from document"
+
+[structure.ContactInfo.fields]
+name = "Full name of contact"
+email = "Email address"
+phone = "Phone number"
+company = "Company or organization name"
+address = "Physical address"
+```
+
+### Analysis Results
+```toml
+[enum.Sentiment]
+definition = "Sentiment analysis results"
+values = ["positive", "negative", "neutral"]
+
+[structure.TextAnalysis]
+definition = "Results of text analysis"
+
+[structure.TextAnalysis.fields]
+sentiment = { type = "Sentiment", definition = "Overall sentiment", required = true }
+key_themes = { type = "list", item_type = "text", definition = "Main themes identified" }
+confidence = { type = "number", definition = "Analysis confidence (0.0-1.0)" }
+word_count = { type = "integer", definition = "Total word count" }
+```
+
+### Classification Results
+```toml
+[structure.Classification]
+definition = "Multi-class classification result"
+
+[structure.Classification.fields]
+predicted_class = { type = "text", definition = "Most likely class", required = true }
+confidence_scores = { type = "dict", key_type = "text", value_type = "number", definition = "Confidence for each class" }
+threshold_met = { type = "boolean", definition = "Whether prediction meets confidence threshold" }
+```
+
+This structured approach ensures your pipeline generates well-validated, type-safe data that can be reliably processed by downstream systems.
diff --git a/pipelex/core/concept_factory.py b/pipelex/core/concept_factory.py
index 1f5cf7183..19d1e6a53 100644
--- a/pipelex/core/concept_factory.py
+++ b/pipelex/core/concept_factory.py
@@ -8,6 +8,7 @@ from pipelex.core.concept_code_factory import ConceptCodeFactory
from pipelex.core.concept_native import NativeConcept, NativeConceptClass
from pipelex.core.domain import SpecialDomain
from pipelex.core.stuff_content import TextContent
+from pipelex.create.structured_output_generator import generate_structured_output_from_inline_definition
from pipelex.exceptions import ConceptFactoryError, StructureClassError
from pipelex.hub import get_class_registry
@@ -16,7 +17,7 @@ class ConceptBlueprint(BaseModel):
model_config = ConfigDict(extra="forbid")
definition: str
- structure: Optional[str] = None
+ structure: Optional[Union[str, Dict[str, Any]]] = None
refines: Union[str, List[str]] = Field(default_factory=list)
domain: Optional[str] = None
@@ -123,12 +124,34 @@ class ConceptFactory:
) -> Concept:
structure_class_name: str
if structure := concept_blueprint.structure:
- # structure is set explicitly
- if not Concept.is_valid_structure_class(structure_class_name=structure):
- raise StructureClassError(
- f"Structure class '{structure}' set for concept '{code}' in domain '{domain}' is not a registered subclass of StuffContent"
- )
- structure_class_name = structure
+ if isinstance(structure, str):
+ # structure is set explicitly as a class name reference
+ if not Concept.is_valid_structure_class(structure_class_name=structure):
+ raise StructureClassError(
+ f"Structure class '{structure}' set for concept '{code}' in domain '{domain}' is not a registered subclass of StuffContent"
+ )
+ structure_class_name = structure
+ else:
+ # structure is defined inline - generate Python class dynamically
+ structure_class_name = code
+ try:
+ # Generate Python class from inline definition
+ python_code = generate_structured_output_from_inline_definition(
+ class_name=code,
+ fields_def=structure, # type: ignore[arg-type]
+ enums=None, # TODO: Handle enums if needed in the future
+ )
+
+ # Execute the generated Python code to register the class
+ exec_globals: Dict[str, Any] = {}
+ exec(python_code, exec_globals)
+
+ # Get the generated class and register it
+ generated_class = exec_globals[code]
+ get_class_registry().register_class(generated_class)
+
+ except Exception as exc:
+ raise ConceptFactoryError(f"Error generating structure class for concept '{code}' in domain '{domain}': {exc}") from exc
elif Concept.is_valid_structure_class(structure_class_name=code):
# structure is set implicitly, by the concept's code
structure_class_name = code
diff --git a/pipelex/create/structured_output_generator.py b/pipelex/create/structured_output_generator.py
index cff3738af..f4a75d771 100644
--- a/pipelex/create/structured_output_generator.py
+++ b/pipelex/create/structured_output_generator.py
@@ -46,10 +46,10 @@ class StructuredOutputGenerator:
self.enum_definitions: Dict[str, Dict[str, Any]] = {} # Store enum definitions
def generate_from_toml(self, toml_content: str) -> str:
- """Generate Python module content from TOML structured output definitions.
+ """Generate Python module content from TOML structure and enum definitions.
Args:
- toml_content: TOML content containing schema and enum definitions
+ toml_content: TOML content containing structure and enum definitions
Returns:
Generated Python module content
@@ -62,18 +62,18 @@ class StructuredOutputGenerator:
enum_defs = data["enum"]
for enum_name, enum_def in enum_defs.items(): # type: ignore[attr-defined,union-attr]
self.enum_definitions[str(enum_name)] = dict(enum_def) # type: ignore[arg-type]
- enum_code = self._generate_enum(str(enum_name), dict(enum_def)) # type: ignore[arg-type]
+ enum_code = self.generate_enum(str(enum_name), dict(enum_def)) # type: ignore[arg-type]
enums.append(enum_code)
- # Process schemas
- if "schema" not in data:
- raise ValueError("TOML must contain a 'schema' section")
+ # Process structures
+ if "structure" not in data:
+ raise ValueError("TOML must contain a 'structure' section")
- schemas = data["schema"]
+ structures = data["structure"]
classes: List[str] = []
- for class_name, schema_def in schemas.items(): # type: ignore[attr-defined,union-attr]
- class_code = self._generate_class(str(class_name), dict(schema_def)) # type: ignore[arg-type]
+ for class_name, structure_def in structures.items(): # type: ignore[attr-defined,union-attr]
+ class_code = self.generate_class(str(class_name), dict(structure_def)) # type: ignore[arg-type]
classes.append(class_code)
# Generate the complete module
@@ -90,7 +90,7 @@ class StructuredOutputGenerator:
return f"{imports_section}\n\n\n{definitions_section}\n"
- def _generate_enum(self, enum_name: str, enum_def: Dict[str, Any]) -> str:
+ def generate_enum(self, enum_name: str, enum_def: Dict[str, Any]) -> str:
"""Generate an enum class definition.
Args:
@@ -131,18 +131,18 @@ class StructuredOutputGenerator:
values_code = "\n".join(value_definitions)
return enum_header + "\n" + values_code
- def _generate_class(self, class_name: str, schema_def: Dict[str, Any]) -> str:
+ def generate_class(self, class_name: str, structure_def: Dict[str, Any]) -> str:
"""Generate a single class definition.
Args:
class_name: Name of the class
- schema_def: Schema definition from TOML
+ structure_def: Structure definition from TOML
Returns:
Generated class code
"""
- definition = schema_def.get("definition", f"Generated {class_name} class")
- fields = schema_def.get("fields", {})
+ definition = structure_def.get("definition", f"Generated {class_name} class")
+ fields = structure_def.get("fields", {})
# Generate class header
class_header = f'class {class_name}(StructuredContent):\n """{definition}"""\n'
@@ -283,7 +283,7 @@ def generate_structured_outputs_from_toml_file(toml_file_path: str, output_file_
"""Generate structured output Python module from TOML file.
Args:
- toml_file_path: Path to input TOML file
+ toml_file_path: Path to input TOML file containing structure definitions
output_file_path: Path to output Python file
"""
with open(toml_file_path, "r", encoding="utf-8") as f:
@@ -300,10 +300,51 @@ def generate_structured_outputs_from_toml_string(toml_content: str) -> str:
"""Generate structured output Python code from TOML string.
Args:
- toml_content: TOML content as string
+ toml_content: TOML content as string containing structure definitions
Returns:
Generated Python module content
"""
generator = StructuredOutputGenerator()
return generator.generate_from_toml(toml_content)
+
+
+def generate_structured_output_from_inline_definition(
+ class_name: str, fields_def: Dict[str, Any], enums: Optional[Dict[str, Dict[str, Any]]] = None
+) -> str:
+ """Generate structured output Python code from inline field definitions.
+
+ Args:
+ class_name: Name of the class to generate
+ fields_def: Dictionary of field definitions (same format as TOML structure.fields)
+ enums: Optional dictionary of enum definitions to include
+
+ Returns:
+ Generated Python module content
+ """
+ generator = StructuredOutputGenerator()
+
+ # Add any provided enums
+ if enums:
+ for enum_name, enum_def in enums.items():
+ generator.enum_definitions[enum_name] = enum_def
+
+ # Create a structure definition from the inline fields
+ structure_def = {"definition": f"Generated {class_name} structure", "fields": fields_def}
+
+ # Generate the class
+ class_code = generator.generate_class(class_name, structure_def)
+
+ # Generate enums if any
+ enum_codes: List[str] = []
+ if enums:
+ for enum_name, enum_def in enums.items():
+ enum_code = generator.generate_enum(enum_name, enum_def)
+ enum_codes.append(enum_code)
+
+ # Combine everything
+ imports_section = "\n".join(sorted(generator.imports))
+ all_definitions: List[str] = enum_codes + [class_code] if enum_codes else [class_code]
+ definitions_section = "\n\n\n".join(all_definitions)
+
+ return f"{imports_section}\n\n\n{definitions_section}\n"
diff --git a/tests/integration/pipelex/test_inline_structure_concepts.py b/tests/integration/pipelex/test_inline_structure_concepts.py
new file mode 100644
index 000000000..ee6c326fd
--- /dev/null
+++ b/tests/integration/pipelex/test_inline_structure_concepts.py
@@ -0,0 +1,178 @@
+"""Integration tests for inline structure definitions in concepts."""
+
+import pytest
+
+from pipelex.core.concept_factory import ConceptBlueprint, ConceptFactory
+from pipelex.core.stuff_content import StructuredContent
+from pipelex.exceptions import ConceptFactoryError, StructureClassError
+from pipelex.hub import get_class_registry
+
+
+class TestInlineStructureConcepts:
+ """Test inline structure definitions in concept blueprints."""
+
+ def test_inline_structure_definition_creation(self):
+ """Test that concepts with inline structure definitions are created correctly."""
+ # Define inline structure with mixed syntax
+ inline_structure = {
+ "dominant_feature": {"type": "text", "definition": "The most important feature", "required": True},
+ "visual_elements": {"type": "text", "definition": "Key visual elements", "required": True},
+ "composition": "Analysis of the image composition",
+ "color_palette": "Description of the main colors",
+ "mood_atmosphere": "The overall mood or atmosphere",
+ }
+
+ # Create concept blueprint with inline structure
+ blueprint = ConceptBlueprint(definition="Analysis of a photo's visual content", structure=inline_structure)
+
+ # Create concept from blueprint
+ concept = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="TestFeatureAnalysis", concept_blueprint=blueprint)
+
+ # Verify concept properties
+ assert concept.code == "test_domain.TestFeatureAnalysis"
+ assert concept.definition == "Analysis of a photo's visual content"
+ assert concept.structure_class_name == "TestFeatureAnalysis"
+ assert concept.domain == "test_domain"
+
+ # Verify the generated class is registered and accessible
+ assert get_class_registry().has_class("TestFeatureAnalysis")
+ generated_class = get_class_registry().get_required_subclass("TestFeatureAnalysis", StructuredContent)
+
+ # Verify class structure
+ assert issubclass(generated_class, StructuredContent)
+ assert generated_class.__name__ == "TestFeatureAnalysis"
+
+ # Test instantiation of generated class
+ instance = generated_class(
+ dominant_feature="A bright red car", # pyright: ignore[reportCallIssue]
+ visual_elements="Car, road, trees, sky", # pyright: ignore[reportCallIssue]
+ composition="Central composition with car in focus", # pyright: ignore[reportCallIssue]
+ color_palette="Red, green, blue, white", # pyright: ignore[reportCallIssue]
+ )
+
+ assert instance.dominant_feature == "A bright red car" # type: ignore
+ assert instance.visual_elements == "Car, road, trees, sky" # type: ignore
+ assert instance.composition == "Central composition with car in focus" # type: ignore
+ assert instance.color_palette == "Red, green, blue, white" # type: ignore
+ assert instance.mood_atmosphere is None # type: ignore
+
+ def test_string_reference_structure_definition(self):
+ """Test that string reference structure definitions still work."""
+ # Create blueprint with string reference
+ blueprint = ConceptBlueprint(definition="Test with string reference", structure="TextContent")
+
+ # Create concept from blueprint
+ concept = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="TestStringRef", concept_blueprint=blueprint)
+
+ # Verify concept properties
+ assert concept.code == "test_domain.TestStringRef"
+ assert concept.definition == "Test with string reference"
+ assert concept.structure_class_name == "TextContent"
+
+ def test_auto_detection_structure(self):
+ """Test auto-detection when no structure is specified."""
+ # Create blueprint without structure
+ blueprint = ConceptBlueprint(definition="Test auto-detection")
+
+ # Create concept from blueprint
+ concept = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="TestAutoDetect", concept_blueprint=blueprint)
+
+ # Should default to TextContent since TestAutoDetect is not a registered class
+ assert concept.structure_class_name == "TextContent"
+
+ def test_inline_structure_with_complex_types(self):
+ """Test inline structure with complex field types."""
+ inline_structure = {
+ "title": {"type": "text", "definition": "Document title", "required": True},
+ "tags": {"type": "list", "item_type": "text", "definition": "List of tags"},
+ "metadata": {"type": "dict", "key_type": "text", "value_type": "text", "definition": "Metadata dictionary"},
+ "priority": {"choices": ["low", "medium", "high"], "definition": "Priority level"},
+ "page_count": {"type": "integer", "definition": "Number of pages"},
+ "is_active": {"type": "boolean", "definition": "Whether document is active"},
+ }
+
+ blueprint = ConceptBlueprint(definition="Complex document structure", structure=inline_structure)
+
+ concept = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="ComplexDocument", concept_blueprint=blueprint)
+
+ # Verify concept creation
+ assert concept.code == "test_domain.ComplexDocument"
+ assert concept.structure_class_name == "ComplexDocument"
+
+ # Verify the generated class works
+ generated_class = get_class_registry().get_required_subclass("ComplexDocument", StructuredContent)
+
+ # Test instantiation with complex types
+ instance = generated_class(
+ title="Test Document",
+ tags=["test", "document"],
+ metadata={"author": "Test Author", "version": "1.0"},
+ priority="high",
+ page_count=42,
+ is_active=True,
+ )
+
+ assert instance.title == "Test Document"
+ assert instance.tags == ["test", "document"]
+ assert instance.metadata == {"author": "Test Author", "version": "1.0"}
+ assert instance.priority == "high"
+ assert instance.page_count == 42
+ assert instance.is_active is True
+
+ def test_invalid_string_reference_raises_error(self):
+ """Test that invalid string references raise appropriate errors."""
+ blueprint = ConceptBlueprint(definition="Test invalid reference", structure="NonExistentClass")
+
+ with pytest.raises(StructureClassError, match="Structure class 'NonExistentClass'"):
+ ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="TestInvalid", concept_blueprint=blueprint)
+
+ def test_inline_structure_generation_error_handling(self):
+ """Test error handling when inline structure generation fails."""
+ # Create invalid structure definition that should cause generation to fail
+ invalid_structure = {
+ "invalid_field": {"type": "nonexistent_type", "definition": "This should fail"},
+ }
+
+ blueprint = ConceptBlueprint(definition="Test error handling", structure=invalid_structure)
+
+ with pytest.raises(ConceptFactoryError, match="Error generating structure class"):
+ ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="TestError", concept_blueprint=blueprint)
+
+ def test_multiple_inline_structures_do_not_conflict(self):
+ """Test that multiple inline structures with same field names don't conflict."""
+ # First structure
+ structure1 = {
+ "name": {"type": "text", "definition": "Person name", "required": True},
+ "age": {"type": "integer", "definition": "Person age"},
+ }
+
+ blueprint1 = ConceptBlueprint(definition="Person information", structure=structure1)
+ concept1 = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="Person", concept_blueprint=blueprint1)
+
+ # Second structure with same field names but different context
+ structure2 = {
+ "name": {"type": "text", "definition": "Product name", "required": True},
+ "age": {"type": "integer", "definition": "Product age in days"},
+ }
+
+ blueprint2 = ConceptBlueprint(definition="Product information", structure=structure2)
+ concept2 = ConceptFactory.make_concept_from_blueprint(domain="test_domain", code="Product", concept_blueprint=blueprint2)
+
+ # Both should be created successfully
+ assert concept1.structure_class_name == "Person"
+ assert concept2.structure_class_name == "Product"
+
+ # Both classes should be registered and independent
+ person_class = get_class_registry().get_required_subclass("Person", StructuredContent)
+ product_class = get_class_registry().get_required_subclass("Product", StructuredContent)
+
+ assert person_class != product_class
+
+ # Test instances are independent
+ person = person_class(name="John Doe", age=30)
+ product = product_class(name="Widget", age=365)
+
+ assert person.name == "John Doe"
+ assert product.name == "Widget"
+ assert person.age == 30
+ assert product.age == 365
diff --git a/tests/unit/pipelex/create/test_structured_output_generator.py b/tests/unit/pipelex/create/test_structured_output_generator.py
index 6cdbc7678..193e85fce 100644
--- a/tests/unit/pipelex/create/test_structured_output_generator.py
+++ b/tests/unit/pipelex/create/test_structured_output_generator.py
@@ -6,18 +6,19 @@ from pipelex import pretty_print
from pipelex.create.structured_output_generator import (
FieldType,
StructuredOutputGenerator,
+ generate_structured_output_from_inline_definition,
generate_structured_outputs_from_toml_string,
)
class TestStructuredOutputGenerator:
- def test_simple_schema_generation(self):
- """Test generation of a simple schema with basic fields."""
+ def test_simple_structure_generation(self):
+ """Test generation of a simple structure with basic fields."""
toml_content = """
-[schema.TestModel]
+[structure.TestModel]
definition = "A test model"
-[schema.TestModel.fields]
+[structure.TestModel.fields]
name = { type = "text", definition = "Name field", required = true }
age = { type = "integer", definition = "Age field" }
active = { type = "boolean", definition = "Active status", default = true }
@@ -42,10 +43,10 @@ active = { type = "boolean", definition = "Active status", default = true }
def test_simplified_field_syntax(self):
"""Test the simplified field syntax where a field is just a string definition."""
toml_content = """
-[schema.SimpleModel]
+[structure.SimpleModel]
definition = "A model with simplified field syntax"
-[schema.SimpleModel.fields]
+[structure.SimpleModel.fields]
dominant_feature = "The most important or dominant feature in the image"
simple_text = "Just some text"
"""
@@ -62,10 +63,10 @@ simple_text = "Just some text"
def test_complex_types_generation(self):
"""Test generation with complex types like lists and dicts."""
toml_content = """
-[schema.ComplexModel]
+[structure.ComplexModel]
definition = "A model with complex types"
-[schema.ComplexModel.fields]
+[structure.ComplexModel.fields]
tags = { type = "list", item_type = "text", definition = "List of tags" }
metadata = { type = "dict", key_type = "text", value_type = "text", definition = "Metadata dictionary" }
scores = { type = "list", item_type = "number", definition = "List of scores", required = true }
@@ -81,19 +82,19 @@ scores = { type = "list", item_type = "number", definition = "List of scores", r
assert 'metadata: Optional[Dict[str, str]] = Field(default=None, description="Metadata dictionary")' in result
assert 'scores: List[float] = Field(..., description="List of scores")' in result
- def test_multiple_schemas(self):
- """Test generation of multiple schemas in one TOML."""
+ def test_multiple_structures(self):
+ """Test generation of multiple structures in one TOML."""
toml_content = """
-[schema.FirstModel]
+[structure.FirstModel]
definition = "First model"
-[schema.FirstModel.fields]
+[structure.FirstModel.fields]
title = { type = "text", definition = "Title", required = true }
-[schema.SecondModel]
+[structure.SecondModel]
definition = "Second model"
-[schema.SecondModel.fields]
+[structure.SecondModel.fields]
content = { type = "text", definition = "Content" }
"""
@@ -108,10 +109,10 @@ content = { type = "text", definition = "Content" }
assert '"""First model"""' in result
assert '"""Second model"""' in result
- def test_empty_schema(self):
- """Test generation of schema with no fields."""
+ def test_empty_structure(self):
+ """Test generation of structure with no fields."""
toml_content = """
-[schema.EmptyModel]
+[structure.EmptyModel]
definition = "An empty model"
"""
@@ -128,10 +129,10 @@ definition = "An empty model"
def test_required_field_syntax(self):
"""Test that fields can be marked as required."""
toml_content = """
-[schema.RequiredFieldsModel]
+[structure.RequiredFieldsModel]
definition = "Model with required fields"
-[schema.RequiredFieldsModel.fields]
+[structure.RequiredFieldsModel.fields]
title = { type = "text", definition = "Required title", required = true }
optional_field = { type = "text", definition = "Optional field" }
"""
@@ -148,10 +149,10 @@ optional_field = { type = "text", definition = "Optional field" }
def test_custom_types(self):
"""Test handling of custom/unknown types."""
toml_content = """
-[schema.CustomTypeModel]
+[structure.CustomTypeModel]
definition = "Model with custom types"
-[schema.CustomTypeModel.fields]
+[structure.CustomTypeModel.fields]
custom_field = { type = "CustomClass", definition = "Custom class field" }
"""
@@ -163,8 +164,8 @@ custom_field = { type = "CustomClass", definition = "Custom class field" }
assert 'custom_field: Optional[CustomClass] = Field(default=None, description="Custom class field")' in result
- def test_missing_schema_section_raises_error(self):
- """Test that missing schema section raises appropriate error."""
+ def test_missing_structure_section_raises_error(self):
+ """Test that missing structure section raises appropriate error."""
toml_content = """
[other_section]
value = "test"
@@ -174,16 +175,16 @@ value = "test"
pretty_print(toml_content, title="Source TOML (should raise error)")
- with pytest.raises(ValueError, match="TOML must contain a 'schema' section"):
+ with pytest.raises(ValueError, match="TOML must contain a 'structure' section"):
generator.generate_from_toml(toml_content)
def test_generate_from_string_convenience_function(self):
"""Test the convenience function for generating from string."""
toml_content = """
-[schema.ConvenienceTest]
+[structure.ConvenienceTest]
definition = "Test convenience function"
-[schema.ConvenienceTest.fields]
+[structure.ConvenienceTest.fields]
value = { type = "text", definition = "Test value", required = true }
"""
@@ -198,10 +199,10 @@ value = { type = "text", definition = "Test value", required = true }
def test_field_type_enum_usage(self):
"""Test that FieldType enum values are properly handled."""
toml_content = """
-[schema.TypeMappingTest]
+[structure.TypeMappingTest]
definition = "Test all type mappings"
-[schema.TypeMappingTest.fields]
+[structure.TypeMappingTest.fields]
text_field = { type = "text", definition = "Text field" }
number_field = { type = "number", definition = "Number field" }
integer_field = { type = "integer", definition = "Integer field" }
@@ -226,10 +227,10 @@ dict_field = { type = "dict", key_type = "text", value_type = "integer", definit
def test_mixed_field_syntax(self):
"""Test mixing simplified syntax with full field definitions."""
toml_content = """
-[schema.MixedSyntaxModel]
+[structure.MixedSyntaxModel]
definition = "Model with mixed field syntax"
-[schema.MixedSyntaxModel.fields]
+[structure.MixedSyntaxModel.fields]
# Simplified syntax
simple_description = "A simple text description"
# Full syntax
@@ -254,10 +255,10 @@ another_simple = "Another simple field"
definition = "Possible states of an order"
values = ["pending", "processing", "shipped", "delivered", "cancelled"]
-[schema.Order]
+[structure.Order]
definition = "An order in the system"
-[schema.Order.fields]
+[structure.Order.fields]
order_id = "Unique identifier for the order"
status = { type = "OrderStatus", definition = "Current status of the order", required = true }
customer_name = "Name of the customer"
@@ -295,10 +296,10 @@ medium = "Medium Priority"
high = "High Priority"
urgent = "Urgent - Handle Immediately"
-[schema.Task]
+[structure.Task]
definition = "A task to be completed"
-[schema.Task.fields]
+[structure.Task.fields]
title = "Task title"
priority = { type = "Priority", definition = "How urgent is this task", required = true }
assigned_to = "Person responsible for the task"
@@ -324,10 +325,10 @@ assigned_to = "Person responsible for the task"
def test_inline_choices(self):
"""Test generation with inline choices (Literal type)."""
toml_content = """
-[schema.Product]
+[structure.Product]
definition = "A product in our catalog"
-[schema.Product.fields]
+[structure.Product.fields]
name = "Product name"
category = { choices = ["electronics", "clothing", "food", "books"], definition = "Product category", required = true }
size = { choices = ["XS", "S", "M", "L", "XL"], definition = "Size of the product" }
@@ -355,10 +356,10 @@ values = ["red", "blue", "green", "yellow", "black", "white"]
definition = "How the product will be shipped"
values = ["standard", "express", "overnight", "pickup"]
-[schema.ProductVariant]
+[structure.ProductVariant]
definition = "A specific variant of a product"
-[schema.ProductVariant.fields]
+[structure.ProductVariant.fields]
sku = "Stock keeping unit"
color = { type = "Color", definition = "Color of this variant" }
size = { choices = ["S", "M", "L"], definition = "Size options" }
@@ -392,10 +393,10 @@ availability = { choices = ["in_stock", "out_of_stock", "pre_order"], definition
definition = "Dietary restrictions and preferences"
values = ["vegetarian", "vegan", "gluten_free", "dairy_free", "nut_free"]
-[schema.MenuItem]
+[structure.MenuItem]
definition = "An item on the restaurant menu"
-[schema.MenuItem.fields]
+[structure.MenuItem.fields]
name = "Name of the dish"
dietary_info = { type = "list", item_type = "DietaryRestriction", definition = "Dietary accommodations" }
tags = { type = "list", item_type = "text", definition = "General tags" }
@@ -415,3 +416,41 @@ tags = { type = "list", item_type = "text", definition = "General tags" }
# Check list with enum type
assert 'dietary_info: Optional[List[DietaryRestriction]] = Field(default=None, description="Dietary accommodations")' in result
assert 'tags: Optional[List[str]] = Field(default=None, description="General tags")' in result
+
+ def test_inline_structure_definition(self):
+ """Test generating structure from inline field definitions (for concept integration)."""
+ fields_def = {
+ "document_type": {"type": "DocumentType", "definition": "Classified document type", "required": True},
+ "priority": {"type": "Priority", "definition": "Processing priority", "required": True},
+ "title": "Inferred document title",
+ "summary": "Brief document summary",
+ "page_count": {"type": "integer", "definition": "Total number of pages"},
+ "language": {"choices": ["en", "fr", "es", "de"], "definition": "Document language"},
+ }
+
+ enums = {
+ "DocumentType": {
+ "definition": "Types of documents we can process",
+ "values": ["technical", "legal", "financial", "marketing", "general"],
+ },
+ "Priority": {"definition": "Document processing priority", "values": ["low", "medium", "high", "urgent"]},
+ }
+
+ result = generate_structured_output_from_inline_definition("DocumentMetadata", fields_def, enums)
+
+ pretty_print(fields_def, title="Inline Field Definition")
+ pretty_print(result, title="Generated Result")
+
+ # Check enum generation
+ assert "class DocumentType(str, Enum):" in result
+ assert "class Priority(str, Enum):" in result
+ assert 'TECHNICAL = "technical"' in result
+ assert 'LOW = "low"' in result
+
+ # Check structure generation
+ assert "class DocumentMetadata(StructuredContent):" in result
+ assert 'document_type: DocumentType = Field(..., description="Classified document type")' in result
+ assert 'priority: Priority = Field(..., description="Processing priority")' in result
+ assert 'title: Optional[str] = Field(default=None, description="Inferred document title")' in result
+ assert 'page_count: Optional[int] = Field(default=None, description="Total number of pages")' in result
+ assert "language: Optional[Literal['en', 'fr', 'es', 'de']] = Field(default=None, description=\"Document language\")" in result