Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Output text for agents: raw TOML, markdown format, shared string utils (#778)
Louis Choquelc11e48be6da6
diff --git a/HANDOFF_SKILLS_OUTPUT_FORMAT.md b/HANDOFF_SKILLS_OUTPUT_FORMAT.mdnew file mode 100644index 000000000..238f5828f--- /dev/null+++ b/HANDOFF_SKILLS_OUTPUT_FORMAT.md@@ -0,0 +1,54 @@+# Handoff: Agent CLI Output Format Changes → Skills Repo++**Date**: 2026-03-23+**PR**: #778 (feature/Output-text-for-agents → dev)+**Scope**: `../skills/skills/` (the published MTHDS marketplace plugin)++## What changed++The `pipelex-agent` CLI changed its output format for several commands:++| Command | Before | After |+|---------|--------|-------|+| `concept` | JSON with `toml` field | Raw TOML to stdout |+| `pipe` | JSON with `toml` field | Raw TOML to stdout |+| `models` | JSON | **Markdown by default**, JSON with `--format json` |+| `doctor` | JSON | **Markdown by default**, JSON with `--format json` |+| `assemble` | JSON | **Removed entirely** |++## What the skills should do++**Do NOT add `--format json` to `models` or `doctor` calls.** The markdown default is intentional — LLM agents consume markdown more naturally than JSON. The skills should expect and work with the markdown output as-is.++The `concept` and `pipe` commands now return raw TOML directly. Any skill that was parsing JSON to extract the `toml` field should read stdout directly instead.++## Files to update++### `skills/shared/mthds-agent-guide.md`++- **Line 47**: "outputs structured JSON (stdout=success, stderr=error with exit code 1)" → Update to reflect the three output modes: JSON for run/validate/inputs/init, raw TOML for concept/pipe, markdown for models/doctor. Errors are always JSON on stderr.+- **Lines 281-282**: Command table entries for `models` and `doctor` — note markdown default output.++### `skills/mthds-build/SKILL.md`++- **Lines 254-257**: `mthds-agent pipelex models --type ...` — these calls now return markdown. No code change needed, but any surrounding text that says "parse the JSON" should be updated.++### `skills/mthds-build/references/talents-and-presets.md` and `skills/mthds-edit/references/talents-and-presets.md`++- **Line 7**: References `mthds-agent pipelex models` — output is now markdown.++### `skills/mthds-pipelex-setup/SKILL.md`++- **Lines 57, 119**: References `mthds-agent pipelex doctor` — output is now markdown.++### `skills/mthds-run/SKILL.md`++- **Line 62**: References `mthds-agent pipelex doctor` — output is now markdown.++### `skills/shared/error-handling.md`++- **Lines 105-106, 115**: These reference `doctor` in error hints/instructions — no change needed, they just tell the agent to run the command.++## Summary++The key message: **models and doctor now speak markdown to the agent, not JSON. This is by design. Do not revert to JSON.**diff --git a/docs/features/cli.md b/docs/features/cli.mdindex 77e171726..5eb632e38 100644--- a/docs/features/cli.md+++ b/docs/features/cli.md@@ -51,11 +51,10 @@ The `pipelex-agent` CLI is a machine-first interface designed for automated envi| `fmt` | Format `.mthds`, `.toml`, or `.plx` files in-place || `lint` | Lint files for errors || `inputs` | Generate example input JSON for a pipe |-| `concept` | Convert a JSON concept spec into TOML |-| `pipe` | Convert a JSON pipe spec into TOML |-| `assemble` | Combine TOML parts into a complete `.mthds` file |-| `models` | List available model presets, aliases, and talent mappings |-| `doctor` | Check configuration health and auto-fix issues |+| `concept` | Convert a JSON concept spec into raw TOML (stdout) |+| `pipe` | Convert a JSON pipe spec into raw TOML (stdout) |+| `models` | List available model presets, aliases, and talent mappings (`--format markdown\|json`, default: markdown) |+| `doctor` | Check configuration health (`--format markdown\|json`, default: markdown) |For detailed CLI documentation, see the [CLI reference](../tools/cli/index.md).diff --git a/docs/tools/cli/agent-cli.md b/docs/tools/cli/agent-cli.mdindex 36f4834eb..6c9a38e9d 100644--- a/docs/tools/cli/agent-cli.md+++ b/docs/tools/cli/agent-cli.md@@ -87,17 +87,21 @@ These commands do not have subcommands:|---------|-------------|| `fmt` | Format a `.mthds`/`.toml`/`.plx` file in-place (delegates to `plxt`) || `lint` | Lint a `.mthds`/`.toml`/`.plx` file for errors (delegates to `plxt`) |-| `concept` | Convert a JSON concept spec into TOML |-| `pipe` | Convert a JSON pipe spec into TOML |-| `assemble` | Merge concept and pipe TOML sections into a complete `.mthds` file |-| `models` | List available model presets, aliases, waterfalls, and talent mappings |-| `doctor` | Check config, credentials, and model health |+| `concept` | Convert a JSON concept spec into raw TOML (stdout) |+| `pipe` | Convert a JSON pipe spec into raw TOML (stdout) |+| `models` | List available model presets, aliases, waterfalls, and talent mappings (`--format markdown\|json`, default: markdown) |+| `doctor` | Check config, credentials, and model health (`--format markdown\|json`, default: markdown) |## Output Contract-Every command (except `fmt` and `lint`) returns structured JSON.+Commands use different stdout formats depending on their purpose:-**Success** — written to stdout:+- **JSON**: `run`, `validate`, `inputs`, `init` — structured JSON via `agent_success()`+- **Raw TOML**: `concept`, `pipe` — TOML text printed directly to stdout+- **Markdown or JSON**: `models`, `doctor` — markdown by default, JSON with `--format json`+- **Passthrough**: `fmt`, `lint` — raw `plxt` output++**JSON success** — written to stdout:```json{@@ -107,7 +111,7 @@ Every command (except `fmt` and `lint`) returns structured JSON.}```-Successful commands return the result object directly. They are not wrapped in a `status` or `data` envelope.+JSON commands return the result object directly. They are not wrapped in a `status` or `data` envelope.**Error** — written to stderr:diff --git a/pipelex/cli/agent_cli/CLAUDE.md b/pipelex/cli/agent_cli/CLAUDE.mdindex 2e8afcaeb..21d680dae 100644--- a/pipelex/cli/agent_cli/CLAUDE.md+++ b/pipelex/cli/agent_cli/CLAUDE.md@@ -45,9 +45,8 @@ commands/fmt_cmd.py # fmt — format file via plxt passthroughlint_cmd.py # lint — lint file via plxt passthroughplxt_passthrough.py # Shared helper for plxt subprocess delegation- concept_cmd.py # concept — JSON spec → concept TOML- pipe_cmd.py # pipe — JSON spec → pipe TOML- assemble_cmd.py # assemble — combine TOML parts into .mthds+ concept_cmd.py # concept — JSON spec → raw TOML to stdout+ pipe_cmd.py # pipe — JSON spec → raw TOML to stdoutmodels_cmd.py # models — list presets, aliases, talent mappingsinit_cmd.py # init — non-interactive config setup (--global/-g, --config/-c)doctor_cmd.py # doctor — config health check (--global/-g)@@ -63,15 +62,14 @@ commands/| `fmt` | Formats a .mthds/.toml/.plx file in-place (delegates to plxt) || `lint` | Lints a .mthds/.toml/.plx file for errors (delegates to plxt) || `inputs` | Generates example input JSON for a pipe/bundle/method (pipe\|bundle\|method subcommands) |-| `concept` | Converts a JSON concept spec into TOML |-| `pipe` | Converts a JSON pipe spec (typed) into TOML |-| `assemble` | Merges concept + pipe TOML sections into a complete `.mthds` file |-| `models` | Lists available model presets, aliases, waterfalls, and talent mappings |-| `doctor` | Checks config, credentials, models health. Use `--global`/`-g` to check `~/.pipelex/` instead of auto-detected config dir. |+| `concept` | Converts a JSON concept spec into raw TOML (stdout) |+| `pipe` | Converts a JSON pipe spec (typed) into raw TOML (stdout) |+| `models` | Lists available model presets, aliases, waterfalls, and talent mappings. `--format markdown\|json` (default: markdown) |+| `doctor` | Checks config, credentials, models health. `--global`/`-g` for global dir. `--format markdown\|json` (default: markdown) |## Key Patterns-- **Output contract**: Every command returns via `agent_success(dict)` or `agent_error(message, error_type, cause)`. Never print outside these. Exception: `fmt` and `lint` are raw passthrough to `plxt` — they bypass `agent_success()`/`agent_error()` intentionally so the calling tool can parse native output.+- **Output contract**: Most commands return via `agent_success(dict)` or `agent_error(message, error_type, cause)`. Exceptions that print directly to stdout: `fmt`/`lint` (plxt passthrough), `concept`/`pipe` (raw TOML), `models`/`doctor` in markdown mode. Errors always go via `agent_error()` regardless of format.- **Error classification**: Each error type maps to a domain (`input`, `config`, `runtime`), a hint string, and a `retryable` flag. See `AGENT_ERROR_HINTS` dict in `agent_output.py`.- **Init**: All commands that need Pipelex use `make_pipelex_for_agent_cli(library_dirs)`. It catches init errors and routes them through `agent_error()`.- **Async core**: Run and validate are async — commands use `asyncio.run()`.diff --git a/pipelex/cli/agent_cli/_agent_cli.py b/pipelex/cli/agent_cli/_agent_cli.pyindex 09213c23e..d7ed8979d 100644--- a/pipelex/cli/agent_cli/_agent_cli.py+++ b/pipelex/cli/agent_cli/_agent_cli.py@@ -8,7 +8,6 @@ from mthds.runners.types import RunnerTypefrom typer.core import TyperGroupfrom typing_extensions import override-from pipelex.cli.agent_cli.commands.assemble_cmd import assemble_cmdfrom pipelex.cli.agent_cli.commands.concept_cmd import concept_cmdfrom pipelex.cli.agent_cli.commands.doctor_cmd import agent_doctor_cmdfrom pipelex.cli.agent_cli.commands.fmt_cmd import fmt_cmd@@ -37,7 +36,6 @@ class PipelexAgentCLI(TyperGroup):"inputs","concept","pipe",- "assemble","models","doctor",]@@ -128,6 +126,5 @@ app.command(name="lint", help="Lint a .mthds, .toml, or .plx file")(lint_cmd)app.add_typer(inputs_app, name="inputs", help="Generate example input JSON for a pipe")app.command(name="concept", help="Structure a concept from JSON spec and output TOML")(concept_cmd)app.command(name="pipe", help="Structure a pipe from JSON spec and output TOML")(pipe_cmd)-app.command(name="assemble", help="Assemble a complete .mthds bundle from TOML parts")(assemble_cmd)app.command(name="models", help="List available model presets, aliases, and talent mappings")(agent_models_cmd)app.command(name="doctor", help="Check Pipelex configuration health and auto-fix issues")(agent_doctor_cmd)diff --git a/pipelex/cli/agent_cli/commands/agent_output.py b/pipelex/cli/agent_cli/commands/agent_output.pyindex 535e43e13..5d00d4798 100644--- a/pipelex/cli/agent_cli/commands/agent_output.py+++ b/pipelex/cli/agent_cli/commands/agent_output.py@@ -8,6 +8,13 @@ import typerfrom pipelex.pipeline.validate_bundle import ValidateBundleErrorfrom pipelex.tools.misc.json_utils import clean_json_dumps+from pipelex.types import StrEnum+++class CliOutputFormat(StrEnum):+ JSON = "json"+ MARKDOWN = "markdown"+AGENT_ERROR_HINTS: dict[str, str] = {# Model/routing errors@@ -45,9 +52,6 @@ AGENT_ERROR_HINTS: dict[str, str] = {"PipelineRequestError": "Check that pipe_code or mthds_content is provided",# Graph errors"GraphSpecParseError": "Validate graphspec.json structure; ensure it matches the expected GraphSpec schema",- # Builder/assembler errors- "ConceptLoadError": "Check TOML syntax in the concept source file or inline string",- "PipeLoadError": "Check TOML syntax in the pipe source file or inline string",# Input/type errors"JsonTypeError": "Input file must be a JSON object {...}, not an array or scalar value","BundleError": "Bundle must declare a 'main_pipe' or use the --pipe flag to specify which pipe to run",@@ -79,8 +83,6 @@ AGENT_ERROR_DOMAINS: dict[str, str] = {"BundleError": "input","PipelineRequestError": "input","GraphSpecParseError": "input",- "ConceptLoadError": "input",- "PipeLoadError": "input","UnknownCommandError": "input",# config = environment/config changes needed"ClientAuthenticationError": "config",diff --git a/pipelex/cli/agent_cli/commands/assemble_cmd.py b/pipelex/cli/agent_cli/commands/assemble_cmd.pydeleted file mode 100644index fb674318a..000000000--- a/pipelex/cli/agent_cli/commands/assemble_cmd.py+++ /dev/null@@ -1,173 +0,0 @@-"""Agent CLI assemble command - assemble TOML bundle from parts with JSON output."""--# pyright: reportUnknownMemberType=false-# pyright: reportArgumentType=false-# mypy: disable-error-code="arg-type"--from pathlib import Path-from typing import Annotated--import tomlkit-import typer--from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success---def _merge_toml_sections(- base_doc: tomlkit.TOMLDocument,- section_name: str,- content_to_merge: tomlkit.TOMLDocument,-) -> None:- """Merge a section from content into the base document.-- Args:- base_doc: The base TOML document to merge into.- section_name: The name of the section (e.g., "concept", "pipe").- content_to_merge: The content document to merge (tomlkit preserves inline tables).- """- if section_name not in content_to_merge:- return-- section_content = content_to_merge[section_name]- if not section_content:- return-- if section_name not in base_doc:- base_doc.add(section_name, tomlkit.table())-- existing_section = base_doc[section_name]- for key, value in section_content.items(): # type: ignore[union-attr]- existing_section[key] = value # type: ignore[index]---def _load_toml_content(source: str) -> tomlkit.TOMLDocument:- """Load TOML content from either a file path or inline string.-- Uses tomlkit to preserve inline tables and formatting.-- Args:- source: Either a file path or inline TOML string.-- Returns:- Parsed TOML document that preserves formatting.- """- source_path = Path(source)- if source_path.exists() and source_path.is_file():- with open(source_path, encoding="utf-8") as the_file:- return tomlkit.load(the_file)- return tomlkit.parse(source)---def assemble_cmd(- domain: Annotated[- str,- typer.Option("--domain", "-d", help="Domain code for the bundle (snake_case)"),- ],- main_pipe: Annotated[- str,- typer.Option("--main-pipe", "-m", help="Main pipe code for the bundle"),- ],- output: Annotated[- str | None,- typer.Option("--output", "-o", help="Output file path (.mthds). Omit to return TOML in JSON response."),- ] = None,- description: Annotated[- str | None,- typer.Option("--description", help="Description of the bundle"),- ] = None,- system_prompt: Annotated[- str | None,- typer.Option("--system-prompt", help="Default system prompt for LLM pipes"),- ] = None,- concepts: Annotated[- list[str] | None,- typer.Option("--concepts", "-c", help="TOML file(s) or inline TOML containing concept definitions"),- ] = None,- pipes: Annotated[- list[str] | None,- typer.Option("--pipes", "-p", help="TOML file(s) or inline TOML containing pipe definitions"),- ] = None,-) -> None:- """Assemble a complete .mthds bundle from individual TOML parts.-- Combines domain configuration, concepts, and pipes into a single valid- Pipelex bundle file. Each --concepts and --pipes argument can be either- a file path or inline TOML content.-- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.-- Examples:- pipelex-agent assemble --domain my_domain --main-pipe main- --concepts concepts.toml --pipes pipes.toml-- pipelex-agent assemble --domain my_domain --main-pipe main- --concepts concepts.toml --pipes pipes.toml --output bundle.mthds- """- try:- # Create base document with domain header- doc = tomlkit.document()- doc.add("domain", domain)-- if description:- doc.add("description", description)-- if system_prompt:- doc.add("system_prompt", system_prompt)-- doc.add("main_pipe", main_pipe)-- # Process concept sources- if concepts:- for concept_source in concepts:- try:- concept_content = _load_toml_content(concept_source)- _merge_toml_sections(doc, "concept", concept_content)- except FileNotFoundError as exc:- agent_error(f"Concept file not found: {concept_source}", "FileNotFoundError", cause=exc)- except Exception as exc:- agent_error(f"Failed to load concepts from '{concept_source}': {exc}", "ConceptLoadError", cause=exc)-- # Process pipe sources- if pipes:- for pipe_source in pipes:- try:- pipe_content = _load_toml_content(pipe_source)- _merge_toml_sections(doc, "pipe", pipe_content)- except FileNotFoundError as exc:- agent_error(f"Pipe file not found: {pipe_source}", "FileNotFoundError", cause=exc)- except Exception as exc:- agent_error(f"Failed to load pipes from '{pipe_source}': {exc}", "PipeLoadError", cause=exc)-- toml_content = tomlkit.dumps(doc)- if not toml_content.endswith("\n"):- toml_content += "\n"-- if output is None:- # Stdout mode (default): return TOML in JSON response- agent_success(- {- "success": True,- "toml": toml_content,- "domain": domain,- "main_pipe": main_pipe,- }- )- else:- # File mode: write to disk- output_path = Path(output)- output_path.parent.mkdir(parents=True, exist_ok=True)- Path(output_path).write_text(toml_content, encoding="utf-8")- agent_success(- {- "success": True,- "bundle_path": str(output_path.resolve()),- "domain": domain,- "main_pipe": main_pipe,- }- )-- except typer.Exit:- raise-- except Exception as exc:- agent_error(str(exc), type(exc).__name__, cause=exc)diff --git a/pipelex/cli/agent_cli/commands/concept_cmd.py b/pipelex/cli/agent_cli/commands/concept_cmd.pyindex 5b51c3245..dd64700e0 100644--- a/pipelex/cli/agent_cli/commands/concept_cmd.py+++ b/pipelex/cli/agent_cli/commands/concept_cmd.py@@ -1,4 +1,4 @@-"""Agent CLI concept command - structure concepts from JSON specs with JSON/TOML output."""+"""Agent CLI concept command - structure concepts from JSON specs with raw TOML output."""# pyright: reportUnknownMemberType=false@@ -10,7 +10,7 @@ import typerfrom pydantic import ValidationErrorfrom pipelex.builder.concept.concept_spec import ConceptSpec, ConceptStructureSpec-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success+from pipelex.cli.agent_cli.commands.agent_output import agent_errorfrom pipelex.language.toml_string_utils import format_toml_stringfrom pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error_for_agent@@ -146,7 +146,7 @@ def concept_cmd(Takes a concept specification in JSON format and converts it to valid PipelexTOML format. Validates the spec before conversion.- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.+ Outputs raw TOML to stdout on success, JSON to stderr on error with exit code 1.JSON spec format:{@@ -192,13 +192,7 @@ def concept_cmd(concept_spec = _parse_concept_spec_from_json(spec_data)toml_content = _concept_spec_to_toml(concept_spec)- agent_success(- {- "success": True,- "concept_code": concept_spec.the_concept_code,- "toml": toml_content,- }- )+ print(toml_content, end="" if toml_content.endswith("\n") else "\n")except ValidationError as exc:message, details = format_pydantic_validation_error_for_agent(exc)diff --git a/pipelex/cli/agent_cli/commands/doctor_cmd.py b/pipelex/cli/agent_cli/commands/doctor_cmd.pyindex a8f1290d2..ca488d184 100644--- a/pipelex/cli/agent_cli/commands/doctor_cmd.py+++ b/pipelex/cli/agent_cli/commands/doctor_cmd.py@@ -1,10 +1,10 @@-"""Agent CLI doctor command -- JSON health report with no interactive prompts."""+"""Agent CLI doctor command -- health report with no interactive prompts."""from typing import Annotated, Anyimport typer-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, agent_successfrom pipelex.cli.commands.doctor_cmd import (ConfigLocationInfo,check_backend_credentials,@@ -16,6 +16,77 @@ from pipelex.cli.commands.doctor_cmd import (from pipelex.system.configuration.config_loader import config_manager+def _status_icon(healthy: bool) -> str:+ """Return a status emoji: checkmark for healthy, warning for unhealthy."""+ return "\u2705" if healthy else "\u26a0\ufe0f"+++def _format_doctor_markdown(result: dict[str, Any]) -> str:+ """Format the doctor result dict as markdown with bullet points."""+ all_healthy: bool = result["all_healthy"]+ config_location: dict[str, Any] = result["config_location"]+ checks: dict[str, Any] = result["checks"]++ status_text = f"All healthy {_status_icon(True)}" if all_healthy else f"Issues found {_status_icon(False)}"+ location_type = "project-local" if config_location.get("is_project_local") else "global"++ lines: list[str] = [+ "# Pipelex Health Check",+ "",+ f"**Status:** {status_text}",+ f"**Config location:** {config_location['config_dir']} ({location_type})",+ ]++ # Config Files+ config_check = checks["config_files"]+ lines.append(f"\n## Config Files \u2014 {_status_icon(config_check['healthy'])}\n")+ lines.append(config_check["message"])++ # Telemetry+ telemetry_check = checks["telemetry"]+ lines.append(f"\n## Telemetry \u2014 {_status_icon(telemetry_check['healthy'])}\n")+ lines.append(telemetry_check["message"])++ # Backend Credentials+ creds_check = checks["backend_credentials"]+ lines.append(f"\n## Backend Credentials \u2014 {_status_icon(creds_check['healthy'])}\n")+ lines.append(creds_check["message"])+ for backend_entry in creds_check.get("backends", []):+ name = backend_entry["backend_name"]+ if backend_entry["all_credentials_valid"]:+ lines.append(f"- **{name}**: All credentials valid")+ else:+ issues: list[str] = []+ missing = backend_entry.get("missing_vars")+ if missing:+ issues.append(f"missing: {', '.join(missing)}")+ placeholder = backend_entry.get("placeholder_vars")+ if placeholder:+ issues.append(f"placeholder: {', '.join(placeholder)}")+ lines.append(f"- **{name}**: {'; '.join(issues) or 'credentials invalid'}")++ # Models+ models_check = checks["models"]+ lines.append(f"\n## Models \u2014 {_status_icon(models_check['healthy'])}\n")+ lines.append(models_check["message"])+ for file_entry in models_check.get("backend_files", []):+ name = file_entry["backend_name"]+ if file_entry["is_valid"]:+ lines.append(f"- **{name}** (`{file_entry['file_path']}`): Valid")+ else:+ error_msg = file_entry.get("error_message", "unknown error")+ lines.append(f"- **{name}** (`{file_entry['file_path']}`): Invalid \u2014 {error_msg}")++ # Recommended Actions+ recommended = result.get("recommended_actions", [])+ if recommended:+ lines.append("\n## Recommended Actions\n")+ for index, action in enumerate(recommended, 1):+ lines.append(f"{index}. {action}")++ return "\n".join(lines)++def agent_doctor_cmd(global_: Annotated[bool,@@ -25,14 +96,19 @@ def agent_doctor_cmd(help="Force checking the global ~/.pipelex/ directory.",),] = False,+ output_format: Annotated[+ CliOutputFormat,+ typer.Option("--format", help="Output format: markdown (default) or json (structured)"),+ ] = CliOutputFormat.MARKDOWN,) -> None:- """Check Pipelex configuration health and output a JSON report.+ """Check Pipelex configuration health and output a diagnostic report.Unlike the human CLI doctor, this command:- - Outputs structured JSON to stdout (never Rich panels or colors)- Does not offer interactive fixes (diagnostic-only)- Includes ``recommended_actions`` for programmatic remediation+ Default output is markdown; use --format json for structured JSON.+Target directory: auto-detects project .pipelex/ if present, else ~/.pipelex/.Use --global/-g to force checking the global ~/.pipelex/ directory."""@@ -143,4 +219,8 @@ def agent_doctor_cmd(if recommended_actions:result["recommended_actions"] = recommended_actions- agent_success(result)+ match output_format:+ case CliOutputFormat.JSON:+ agent_success(result)+ case CliOutputFormat.MARKDOWN:+ print(_format_doctor_markdown(result))diff --git a/pipelex/cli/agent_cli/commands/models_cmd.py b/pipelex/cli/agent_cli/commands/models_cmd.pyindex d61d14803..2164d3157 100644--- a/pipelex/cli/agent_cli/commands/models_cmd.py+++ b/pipelex/cli/agent_cli/commands/models_cmd.py@@ -5,7 +5,7 @@ from typing import Annotated, Anyimport typerfrom pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_cli-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, agent_successfrom pipelex.cogt.model_backends.model_spec import InferenceModelSpecfrom pipelex.cogt.model_backends.model_type import ModelTypefrom pipelex.cogt.models.model_deck import ModelDeck@@ -223,6 +223,66 @@ def _build_talent_mappings_for_category(return _filter_talent_mappings_by_backend(mappings, presets_dict, model_deck, model_type, backend)+CATEGORY_DISPLAY_NAMES: dict[str, str] = {+ "llm": "LLM",+ "img_gen": "Image Generation",+ "extract": "Extract",+ "search": "Search",+}+++def _format_models_markdown(result: dict[str, Any]) -> str:+ """Format the models result dict as markdown with bullet points."""+ lines: list[str] = ["# Available Models"]++ presets: dict[str, list[dict[str, Any]]] = result["presets"]+ aliases: dict[str, dict[str, str]] = result["aliases"]+ waterfalls: dict[str, dict[str, list[str]]] = result["waterfalls"]+ talent_mappings: dict[str, dict[str, str]] = result["talent_mappings"]++ for category_key, display_name in CATEGORY_DISPLAY_NAMES.items():+ cat_presets = presets.get(category_key, [])+ cat_aliases = aliases.get(category_key, {})+ cat_waterfalls = waterfalls.get(category_key, {})+ cat_talents = talent_mappings.get(category_key, {})++ if not cat_presets and not cat_aliases and not cat_waterfalls and not cat_talents:+ continue++ lines.append(f"\n## {display_name}")++ if cat_presets:+ lines.append("\n### Presets\n")+ for preset in cat_presets:+ description = preset.get("description", "")+ if description:+ lines.append(f"- **{preset['name']}**: {description}")+ else:+ lines.append(f"- **{preset['name']}**")++ if cat_aliases:+ lines.append("\n### Aliases\n")+ for alias_name, alias_target in cat_aliases.items():+ lines.append(f"- **{alias_name}** \u2192 {alias_target}")++ if cat_waterfalls:+ lines.append("\n### Waterfalls\n")+ for waterfall_name, fallback_list in cat_waterfalls.items():+ chain = " \u2192 ".join(fallback_list)+ lines.append(f"- **{waterfall_name}**: {chain}")++ if cat_talents:+ lines.append("\n### Talent Mappings\n")+ for talent_name, preset_name in cat_talents.items():+ lines.append(f"- **{talent_name}** \u2192 {preset_name}")++ hint = result.get("talent_mappings_usage_hint")+ if hint:+ lines.append(f"\n---\n\n{hint}")++ return "\n".join(lines)++def agent_models_cmd(ctx: typer.Context,model_type: Annotated[@@ -233,11 +293,15 @@ def agent_models_cmd(str | None,typer.Option("--backend", "-b", help="Filter by backend name"),] = None,+ output_format: Annotated[+ CliOutputFormat,+ typer.Option("--format", help="Output format: markdown (default) or json (structured)"),+ ] = CliOutputFormat.MARKDOWN,) -> None:"""List available model presets, aliases, waterfalls, and talent mappings.- Outputs structured JSON to stdout with all model configuration- that an agent needs to reference when building pipelines.+ Outputs model configuration that an agent needs to reference when building pipelines.+ Default output is markdown; use --format json for structured JSON."""try:make_pipelex_for_agent_cli(log_level=ctx.obj["log_level"], needs_inference=False, needs_model_specs=backend is not None)@@ -287,7 +351,11 @@ def agent_models_cmd(),}- agent_success(result)+ match output_format:+ case CliOutputFormat.JSON:+ agent_success(result)+ case CliOutputFormat.MARKDOWN:+ print(_format_models_markdown(result))except SystemExit:# agent_error already handled and called sys.exitraisediff --git a/pipelex/cli/agent_cli/commands/pipe_cmd.py b/pipelex/cli/agent_cli/commands/pipe_cmd.pyindex 2fa6db90f..c89599372 100644--- a/pipelex/cli/agent_cli/commands/pipe_cmd.py+++ b/pipelex/cli/agent_cli/commands/pipe_cmd.py@@ -1,4 +1,4 @@-"""Agent CLI pipe command - structure pipes from JSON specs with JSON/TOML output."""+"""Agent CLI pipe command - structure pipes from JSON specs with raw TOML output."""# pyright: reportUnknownMemberType=false# pyright: reportArgumentType=false@@ -27,7 +27,7 @@ from pipelex.builder.pipe.pipe_spec_map import pipe_type_to_spec_classfrom pipelex.builder.talents.extract_talent import ExtractTalentfrom pipelex.builder.talents.img_gen_talent import ImgGenTalentfrom pipelex.builder.talents.llm_talent import LLMTalent-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success+from pipelex.cli.agent_cli.commands.agent_output import agent_errorfrom pipelex.core.pipes.pipe_blueprint import PipeTypefrom pipelex.language.toml_string_utils import format_toml_stringfrom pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error_for_agent@@ -337,7 +337,7 @@ def pipe_cmd(Takes a pipe specification in JSON format and converts it to valid PipelexTOML format. Validates the spec before conversion.- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.+ Outputs raw TOML to stdout on success, JSON to stderr on error with exit code 1.JSON spec format (varies by pipe type):@@ -407,14 +407,7 @@ def pipe_cmd(pipe_spec = _parse_pipe_spec_from_json(resolved_pipe_type, spec_data)toml_content = _pipe_spec_to_toml(pipe_spec)- agent_success(- {- "success": True,- "pipe_code": pipe_spec.pipe_code,- "pipe_type": resolved_pipe_type,- "toml": toml_content,- }- )+ print(toml_content, end="" if toml_content.endswith("\n") else "\n")except ValidationError as exc:message, details = format_pydantic_validation_error_for_agent(exc)diff --git a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py b/tests/unit/pipelex/cli/test_agent_doctor_cmd.pyindex 87df5abf5..f57bcf80e 100644--- a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py+++ b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py@@ -11,6 +11,7 @@ import typerif TYPE_CHECKING:from pytest_mock import MockerFixture+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormatfrom pipelex.cli.agent_cli.commands.doctor_cmd import agent_doctor_cmdfrom pipelex.cogt.model_backends.backend_library import BackendCredentialsReport@@ -37,7 +38,7 @@ class TestAgentDoctorCmd:return_value=(True, "Models valid", {}),)- agent_doctor_cmd()+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert parsed["success"] is True@@ -63,7 +64,7 @@ class TestAgentDoctorCmd:return_value=(True, "Models valid", {}),)- agent_doctor_cmd()+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert parsed["all_healthy"] is False@@ -101,7 +102,7 @@ class TestAgentDoctorCmd:return_value=(True, "Models valid", {}),)- agent_doctor_cmd()+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert parsed["all_healthy"] is Falsediff --git a/tests/unit/pipelex/cli/test_agent_models_cmd.py b/tests/unit/pipelex/cli/test_agent_models_cmd.pyindex b80f2d4a0..56b9c04c8 100644--- a/tests/unit/pipelex/cli/test_agent_models_cmd.py+++ b/tests/unit/pipelex/cli/test_agent_models_cmd.py@@ -9,6 +9,7 @@ if TYPE_CHECKING:import pytestfrom pytest_mock import MockerFixture+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormatfrom pipelex.cli.agent_cli.commands.models_cmd import ModelCategory, agent_models_cmdfrom pipelex.cogt.model_backends.model_type import ModelType@@ -151,7 +152,7 @@ class TestAgentModelsCmd:"""No filters should return all 3 categories in every section."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx)+ agent_models_cmd(ctx=agent_ctx, output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert parsed["success"] is True@@ -164,7 +165,7 @@ class TestAgentModelsCmd:"""No filters should include all preset entries with correct data."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx)+ agent_models_cmd(ctx=agent_ctx, output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)llm_preset_names = [preset["name"] for preset in parsed["presets"]["llm"]]@@ -178,7 +179,7 @@ class TestAgentModelsCmd:"""--type llm should return only llm keys in all sections."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM])+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)for section in ("presets", "aliases", "waterfalls", "talent_mappings"):@@ -190,7 +191,7 @@ class TestAgentModelsCmd:"""--type extract should return only extract keys in all sections."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.EXTRACT])+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.EXTRACT], output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)for section in ("presets", "aliases", "waterfalls", "talent_mappings"):@@ -202,7 +203,7 @@ class TestAgentModelsCmd:"""--type img_gen should return only img_gen keys in all sections."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.IMG_GEN])+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.IMG_GEN], output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)for section in ("presets", "aliases", "waterfalls", "talent_mappings"):@@ -214,7 +215,7 @@ class TestAgentModelsCmd:"""--type llm --type img_gen should return both llm and img_gen, but not extract."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM, ModelCategory.IMG_GEN])+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM, ModelCategory.IMG_GEN], output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)for section in ("presets", "aliases", "waterfalls", "talent_mappings"):@@ -226,7 +227,7 @@ class TestAgentModelsCmd:"""--backend openai should filter presets/aliases/waterfalls to only openai-backed models."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, backend="openai")+ agent_models_cmd(ctx=agent_ctx, backend="openai", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)# LLM: only 'fast' preset (gpt-4o-mini on openai), not 'smart' (claude-sonnet on anthropic)@@ -245,7 +246,7 @@ class TestAgentModelsCmd:"""--backend anthropic should include only anthropic-backed models."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, backend="anthropic")+ agent_models_cmd(ctx=agent_ctx, backend="anthropic", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)llm_preset_names = [preset["name"] for preset in parsed["presets"]["llm"]]@@ -258,7 +259,7 @@ class TestAgentModelsCmd:"""--type llm --backend openai should combine both filters."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], backend="openai")+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], backend="openai", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)# Only llm category@@ -274,7 +275,7 @@ class TestAgentModelsCmd:"""--backend nonexistent should produce empty presets/aliases/waterfalls/talent_mappings."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, backend="nonexistent")+ agent_models_cmd(ctx=agent_ctx, backend="nonexistent", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert parsed["success"] is True@@ -286,7 +287,7 @@ class TestAgentModelsCmd:"""--backend anthropic should include llm waterfall (has claude-sonnet) but not img waterfall."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, backend="anthropic")+ agent_models_cmd(ctx=agent_ctx, backend="anthropic", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert "llm-wf" in parsed["waterfalls"]["llm"]@@ -297,7 +298,7 @@ class TestAgentModelsCmd:"""--backend openai should include LLM talent_mapping (fast->gpt-4o-mini) and img_gen, but not extract."""_setup_mocks(mocker)- agent_models_cmd(ctx=agent_ctx, backend="openai")+ agent_models_cmd(ctx=agent_ctx, backend="openai", output_format=CliOutputFormat.JSON)parsed = json.loads(capsys.readouterr().out)assert "text_gen" in parsed["talent_mappings"]["llm"]diff --git a/tests/unit/pipelex/cli/test_format_doctor_markdown.py b/tests/unit/pipelex/cli/test_format_doctor_markdown.pynew file mode 100644index 000000000..f89c390ec--- /dev/null+++ b/tests/unit/pipelex/cli/test_format_doctor_markdown.py@@ -0,0 +1,142 @@+"""Unit tests for the doctor command markdown formatter."""++from __future__ import annotations++from typing import Any, ClassVar++from pipelex.cli.agent_cli.commands.doctor_cmd import (+ _format_doctor_markdown, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]+)+++class TestFormatDoctorMarkdown:+ """Tests for _format_doctor_markdown output."""++ _HEALTHY_RESULT: ClassVar[dict[str, Any]] = {+ "success": True,+ "all_healthy": True,+ "config_location": {+ "config_dir": "/home/user/.pipelex",+ "is_project_local": False,+ "project_root": None,+ "global_config_dir": "/home/user/.pipelex",+ },+ "checks": {+ "config_files": {"healthy": True, "message": "All config files present", "missing_count": 0},+ "telemetry": {"healthy": True, "message": "Telemetry configured"},+ "backend_credentials": {"healthy": True, "message": "All backends healthy", "backends": []},+ "models": {"healthy": True, "message": "Models valid", "backend_files": []},+ },+ }++ _UNHEALTHY_RESULT: ClassVar[dict[str, Any]] = {+ "success": True,+ "all_healthy": False,+ "config_location": {+ "config_dir": "/project/.pipelex",+ "is_project_local": True,+ "project_root": "/project",+ "global_config_dir": "/home/user/.pipelex",+ },+ "checks": {+ "config_files": {"healthy": True, "message": "OK", "missing_count": 0},+ "telemetry": {"healthy": False, "message": "Config format outdated"},+ "backend_credentials": {+ "healthy": False,+ "message": "1 backend has issues",+ "backends": [+ {"backend_name": "openai", "all_credentials_valid": True},+ {"backend_name": "anthropic", "all_credentials_valid": False, "missing_vars": ["ANTHROPIC_API_KEY"]},+ ],+ },+ "models": {+ "healthy": True,+ "message": "Models valid",+ "backend_files": [+ {"backend_name": "openai", "file_path": ".pipelex/inference/backends/openai.toml", "is_valid": True},+ ],+ },+ },+ "recommended_actions": [+ "Set environment variable: ANTHROPIC_API_KEY",+ ],+ }++ def test_healthy_output_contains_status_and_checkmark(self) -> None:+ """Healthy result should show 'All healthy' with checkmark emoji."""+ output = _format_doctor_markdown(self._HEALTHY_RESULT)+ assert "All healthy \u2705" in output++ def test_healthy_output_shows_global_location(self) -> None:+ """Global config should be labeled as 'global'."""+ output = _format_doctor_markdown(self._HEALTHY_RESULT)+ assert "/home/user/.pipelex (global)" in output++ def test_healthy_output_has_no_recommended_actions(self) -> None:+ """Healthy result should not include a Recommended Actions section."""+ output = _format_doctor_markdown(self._HEALTHY_RESULT)+ assert "Recommended Actions" not in output++ def test_unhealthy_output_shows_warning_status(self) -> None:+ """Unhealthy result should show 'Issues found' with warning emoji."""+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)+ assert "Issues found \u26a0\ufe0f" in output++ def test_unhealthy_output_shows_project_local(self) -> None:+ """Project-local config should be labeled as 'project-local'."""+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)+ assert "/project/.pipelex (project-local)" in output++ def test_unhealthy_output_lists_missing_credentials(self) -> None:+ """Missing credentials should appear in the backend credentials section."""+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)+ assert "ANTHROPIC_API_KEY" in output+ assert "**openai**: All credentials valid" in output++ def test_unhealthy_output_includes_recommended_actions(self) -> None:+ """Recommended actions should appear as a numbered list."""+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)+ assert "1. Set environment variable: ANTHROPIC_API_KEY" in output++ def test_invalid_model_file_shows_error(self) -> None:+ """Invalid model file should show error message in output."""+ result: dict[str, Any] = {+ **self._HEALTHY_RESULT,+ "all_healthy": False,+ "checks": {+ **self._HEALTHY_RESULT["checks"],+ "models": {+ "healthy": False,+ "message": "Issues found",+ "backend_files": [+ {+ "backend_name": "bad_backend",+ "file_path": ".pipelex/inference/backends/bad.toml",+ "is_valid": False,+ "error_message": "Parse error on line 5",+ },+ ],+ },+ },+ }+ output = _format_doctor_markdown(result)+ assert "Invalid \u2014 Parse error on line 5" in output++ def test_credentials_invalid_without_details_shows_fallback(self) -> None:+ """Backend with invalid credentials but no missing/placeholder vars shows fallback text."""+ result: dict[str, Any] = {+ **self._HEALTHY_RESULT,+ "all_healthy": False,+ "checks": {+ **self._HEALTHY_RESULT["checks"],+ "backend_credentials": {+ "healthy": False,+ "message": "Issues",+ "backends": [+ {"backend_name": "mystery", "all_credentials_valid": False},+ ],+ },+ },+ }+ output = _format_doctor_markdown(result)+ assert "**mystery**: credentials invalid" in outputdiff --git a/tests/unit/pipelex/cli/test_format_models_markdown.py b/tests/unit/pipelex/cli/test_format_models_markdown.pynew file mode 100644index 000000000..1fa9365a0--- /dev/null+++ b/tests/unit/pipelex/cli/test_format_models_markdown.py@@ -0,0 +1,125 @@+"""Unit tests for the models command markdown formatter."""++from __future__ import annotations++from typing import Any, ClassVar++from pipelex.cli.agent_cli.commands.models_cmd import (+ _format_models_markdown, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]+)+++class TestFormatModelsMarkdown:+ """Tests for _format_models_markdown output."""++ _FULL_RESULT: ClassVar[dict[str, Any]] = {+ "success": True,+ "presets": {+ "llm": [+ {"name": "writing-creative", "description": "Creative writing"},+ {"name": "engineering-code"},+ ],+ "img_gen": [{"name": "gen-image", "description": "Image generation"}],+ "extract": [],+ "search": [],+ },+ "aliases": {+ "llm": {"fast": "gpt4o-mini", "smart": "claude-sonnet"},+ "img_gen": {},+ "extract": {},+ "search": {},+ },+ "waterfalls": {+ "llm": {"robust": ["gpt4o", "claude-sonnet", "gemini"]},+ "img_gen": {},+ "extract": {},+ "search": {},+ },+ "talent_mappings": {+ "llm": {"creative-writer": "writing-creative"},+ "img_gen": {},+ "extract": {},+ "search": {},+ },+ "talent_mappings_usage_hint": "Use the talent name as the value for llm_talent",+ }++ def test_output_starts_with_heading(self) -> None:+ """Output should start with the main heading."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert output.startswith("# Available Models")++ def test_llm_category_present(self) -> None:+ """LLM category with data should appear as a section."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "## LLM" in output++ def test_preset_with_description(self) -> None:+ """Preset with description should show name and description."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "- **writing-creative**: Creative writing" in output++ def test_preset_without_description(self) -> None:+ """Preset without description should show only the name."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "- **engineering-code**" in output++ def test_aliases_use_arrow(self) -> None:+ """Aliases should use arrow notation."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "- **fast** \u2192 gpt4o-mini" in output++ def test_waterfalls_show_chain(self) -> None:+ """Waterfalls should show the full fallback chain with arrows."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "gpt4o \u2192 claude-sonnet \u2192 gemini" in output++ def test_talent_mappings_present(self) -> None:+ """Talent mappings should use arrow notation."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "- **creative-writer** \u2192 writing-creative" in output++ def test_hint_present(self) -> None:+ """Usage hint should appear at the bottom."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "Use the talent name as the value for llm_talent" in output++ def test_empty_categories_omitted(self) -> None:+ """Categories with no data should not appear in output."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "## Extract" not in output+ assert "## Search" not in output++ def test_img_gen_category_present(self) -> None:+ """Image Generation category with presets should appear."""+ output = _format_models_markdown(self._FULL_RESULT)+ assert "## Image Generation" in output+ assert "- **gen-image**: Image generation" in output++ def test_all_empty_produces_heading_only(self) -> None:+ """Result with all empty sections should produce only the heading and hint."""+ empty_result: dict[str, Any] = {+ "presets": {"llm": [], "img_gen": [], "extract": [], "search": []},+ "aliases": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},+ "waterfalls": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},+ "talent_mappings": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},+ "talent_mappings_usage_hint": "hint text",+ }+ output = _format_models_markdown(empty_result)+ assert "# Available Models" in output+ assert "## LLM" not in output+ assert "hint text" in output++ def test_empty_subsections_omitted_within_category(self) -> None:+ """Within a category, subsections with no data should be omitted."""+ result: dict[str, Any] = {+ "presets": {"llm": [{"name": "only-preset"}]},+ "aliases": {"llm": {}},+ "waterfalls": {"llm": {}},+ "talent_mappings": {"llm": {}},+ }+ output = _format_models_markdown(result)+ assert "### Presets" in output+ assert "### Aliases" not in output+ assert "### Waterfalls" not in output+ assert "### Talent Mappings" not in output