Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Output text for agents: raw TOML, markdown format, shared string utils (#778)

Louis Choquelc11e48be6da6
diff --git a/HANDOFF_SKILLS_OUTPUT_FORMAT.md b/HANDOFF_SKILLS_OUTPUT_FORMAT.md
new file mode 100644
index 000000000..238f5828f
--- /dev/null
+++ b/HANDOFF_SKILLS_OUTPUT_FORMAT.md
@@ -0,0 +1,54 @@
+# Handoff: Agent CLI Output Format Changes → Skills Repo
+
+**Date**: 2026-03-23
+**PR**: #778 (feature/Output-text-for-agents → dev)
+**Scope**: `../skills/skills/` (the published MTHDS marketplace plugin)
+
+## What changed
+
+The `pipelex-agent` CLI changed its output format for several commands:
+
+| Command | Before | After |
+|---------|--------|-------|
+| `concept` | JSON with `toml` field | Raw TOML to stdout |
+| `pipe` | JSON with `toml` field | Raw TOML to stdout |
+| `models` | JSON | **Markdown by default**, JSON with `--format json` |
+| `doctor` | JSON | **Markdown by default**, JSON with `--format json` |
+| `assemble` | JSON | **Removed entirely** |
+
+## What the skills should do
+
+**Do NOT add `--format json` to `models` or `doctor` calls.** The markdown default is intentional — LLM agents consume markdown more naturally than JSON. The skills should expect and work with the markdown output as-is.
+
+The `concept` and `pipe` commands now return raw TOML directly. Any skill that was parsing JSON to extract the `toml` field should read stdout directly instead.
+
+## Files to update
+
+### `skills/shared/mthds-agent-guide.md`
+
+- **Line 47**: "outputs structured JSON (stdout=success, stderr=error with exit code 1)" → Update to reflect the three output modes: JSON for run/validate/inputs/init, raw TOML for concept/pipe, markdown for models/doctor. Errors are always JSON on stderr.
+- **Lines 281-282**: Command table entries for `models` and `doctor` — note markdown default output.
+
+### `skills/mthds-build/SKILL.md`
+
+- **Lines 254-257**: `mthds-agent pipelex models --type ...` — these calls now return markdown. No code change needed, but any surrounding text that says "parse the JSON" should be updated.
+
+### `skills/mthds-build/references/talents-and-presets.md` and `skills/mthds-edit/references/talents-and-presets.md`
+
+- **Line 7**: References `mthds-agent pipelex models` — output is now markdown.
+
+### `skills/mthds-pipelex-setup/SKILL.md`
+
+- **Lines 57, 119**: References `mthds-agent pipelex doctor` — output is now markdown.
+
+### `skills/mthds-run/SKILL.md`
+
+- **Line 62**: References `mthds-agent pipelex doctor` — output is now markdown.
+
+### `skills/shared/error-handling.md`
+
+- **Lines 105-106, 115**: These reference `doctor` in error hints/instructions — no change needed, they just tell the agent to run the command.
+
+## Summary
+
+The key message: **models and doctor now speak markdown to the agent, not JSON. This is by design. Do not revert to JSON.**
diff --git a/docs/features/cli.md b/docs/features/cli.md
index 77e171726..5eb632e38 100644
--- a/docs/features/cli.md
+++ b/docs/features/cli.md
@@ -51,11 +51,10 @@ The `pipelex-agent` CLI is a machine-first interface designed for automated envi
| `fmt` | Format `.mthds`, `.toml`, or `.plx` files in-place |
| `lint` | Lint files for errors |
| `inputs` | Generate example input JSON for a pipe |
-| `concept` | Convert a JSON concept spec into TOML |
-| `pipe` | Convert a JSON pipe spec into TOML |
-| `assemble` | Combine TOML parts into a complete `.mthds` file |
-| `models` | List available model presets, aliases, and talent mappings |
-| `doctor` | Check configuration health and auto-fix issues |
+| `concept` | Convert a JSON concept spec into raw TOML (stdout) |
+| `pipe` | Convert a JSON pipe spec into raw TOML (stdout) |
+| `models` | List available model presets, aliases, and talent mappings (`--format markdown\|json`, default: markdown) |
+| `doctor` | Check configuration health (`--format markdown\|json`, default: markdown) |
For detailed CLI documentation, see the [CLI reference](../tools/cli/index.md).
diff --git a/docs/tools/cli/agent-cli.md b/docs/tools/cli/agent-cli.md
index 36f4834eb..6c9a38e9d 100644
--- a/docs/tools/cli/agent-cli.md
+++ b/docs/tools/cli/agent-cli.md
@@ -87,17 +87,21 @@ These commands do not have subcommands:
|---------|-------------|
| `fmt` | Format a `.mthds`/`.toml`/`.plx` file in-place (delegates to `plxt`) |
| `lint` | Lint a `.mthds`/`.toml`/`.plx` file for errors (delegates to `plxt`) |
-| `concept` | Convert a JSON concept spec into TOML |
-| `pipe` | Convert a JSON pipe spec into TOML |
-| `assemble` | Merge concept and pipe TOML sections into a complete `.mthds` file |
-| `models` | List available model presets, aliases, waterfalls, and talent mappings |
-| `doctor` | Check config, credentials, and model health |
+| `concept` | Convert a JSON concept spec into raw TOML (stdout) |
+| `pipe` | Convert a JSON pipe spec into raw TOML (stdout) |
+| `models` | List available model presets, aliases, waterfalls, and talent mappings (`--format markdown\|json`, default: markdown) |
+| `doctor` | Check config, credentials, and model health (`--format markdown\|json`, default: markdown) |
## Output Contract
-Every command (except `fmt` and `lint`) returns structured JSON.
+Commands use different stdout formats depending on their purpose:
-**Success** — written to stdout:
+- **JSON**: `run`, `validate`, `inputs`, `init` — structured JSON via `agent_success()`
+- **Raw TOML**: `concept`, `pipe` — TOML text printed directly to stdout
+- **Markdown or JSON**: `models`, `doctor` — markdown by default, JSON with `--format json`
+- **Passthrough**: `fmt`, `lint` — raw `plxt` output
+
+**JSON success** — written to stdout:
```json
{
@@ -107,7 +111,7 @@ Every command (except `fmt` and `lint`) returns structured JSON.
}
```
-Successful commands return the result object directly. They are not wrapped in a `status` or `data` envelope.
+JSON commands return the result object directly. They are not wrapped in a `status` or `data` envelope.
**Error** — written to stderr:
diff --git a/pipelex/cli/agent_cli/CLAUDE.md b/pipelex/cli/agent_cli/CLAUDE.md
index 2e8afcaeb..21d680dae 100644
--- a/pipelex/cli/agent_cli/CLAUDE.md
+++ b/pipelex/cli/agent_cli/CLAUDE.md
@@ -45,9 +45,8 @@ commands/
fmt_cmd.py # fmt — format file via plxt passthrough
lint_cmd.py # lint — lint file via plxt passthrough
plxt_passthrough.py # Shared helper for plxt subprocess delegation
- concept_cmd.py # concept — JSON spec → concept TOML
- pipe_cmd.py # pipe — JSON spec → pipe TOML
- assemble_cmd.py # assemble — combine TOML parts into .mthds
+ concept_cmd.py # concept — JSON spec → raw TOML to stdout
+ pipe_cmd.py # pipe — JSON spec → raw TOML to stdout
models_cmd.py # models — list presets, aliases, talent mappings
init_cmd.py # init — non-interactive config setup (--global/-g, --config/-c)
doctor_cmd.py # doctor — config health check (--global/-g)
@@ -63,15 +62,14 @@ commands/
| `fmt` | Formats a .mthds/.toml/.plx file in-place (delegates to plxt) |
| `lint` | Lints a .mthds/.toml/.plx file for errors (delegates to plxt) |
| `inputs` | Generates example input JSON for a pipe/bundle/method (pipe\|bundle\|method subcommands) |
-| `concept` | Converts a JSON concept spec into TOML |
-| `pipe` | Converts a JSON pipe spec (typed) into TOML |
-| `assemble` | Merges concept + pipe TOML sections into a complete `.mthds` file |
-| `models` | Lists available model presets, aliases, waterfalls, and talent mappings |
-| `doctor` | Checks config, credentials, models health. Use `--global`/`-g` to check `~/.pipelex/` instead of auto-detected config dir. |
+| `concept` | Converts a JSON concept spec into raw TOML (stdout) |
+| `pipe` | Converts a JSON pipe spec (typed) into raw TOML (stdout) |
+| `models` | Lists available model presets, aliases, waterfalls, and talent mappings. `--format markdown\|json` (default: markdown) |
+| `doctor` | Checks config, credentials, models health. `--global`/`-g` for global dir. `--format markdown\|json` (default: markdown) |
## Key Patterns
-- **Output contract**: Every command returns via `agent_success(dict)` or `agent_error(message, error_type, cause)`. Never print outside these. Exception: `fmt` and `lint` are raw passthrough to `plxt` — they bypass `agent_success()`/`agent_error()` intentionally so the calling tool can parse native output.
+- **Output contract**: Most commands return via `agent_success(dict)` or `agent_error(message, error_type, cause)`. Exceptions that print directly to stdout: `fmt`/`lint` (plxt passthrough), `concept`/`pipe` (raw TOML), `models`/`doctor` in markdown mode. Errors always go via `agent_error()` regardless of format.
- **Error classification**: Each error type maps to a domain (`input`, `config`, `runtime`), a hint string, and a `retryable` flag. See `AGENT_ERROR_HINTS` dict in `agent_output.py`.
- **Init**: All commands that need Pipelex use `make_pipelex_for_agent_cli(library_dirs)`. It catches init errors and routes them through `agent_error()`.
- **Async core**: Run and validate are async — commands use `asyncio.run()`.
diff --git a/pipelex/cli/agent_cli/_agent_cli.py b/pipelex/cli/agent_cli/_agent_cli.py
index 09213c23e..d7ed8979d 100644
--- a/pipelex/cli/agent_cli/_agent_cli.py
+++ b/pipelex/cli/agent_cli/_agent_cli.py
@@ -8,7 +8,6 @@ from mthds.runners.types import RunnerType
from typer.core import TyperGroup
from typing_extensions import override
-from pipelex.cli.agent_cli.commands.assemble_cmd import assemble_cmd
from pipelex.cli.agent_cli.commands.concept_cmd import concept_cmd
from pipelex.cli.agent_cli.commands.doctor_cmd import agent_doctor_cmd
from pipelex.cli.agent_cli.commands.fmt_cmd import fmt_cmd
@@ -37,7 +36,6 @@ class PipelexAgentCLI(TyperGroup):
"inputs",
"concept",
"pipe",
- "assemble",
"models",
"doctor",
]
@@ -128,6 +126,5 @@ app.command(name="lint", help="Lint a .mthds, .toml, or .plx file")(lint_cmd)
app.add_typer(inputs_app, name="inputs", help="Generate example input JSON for a pipe")
app.command(name="concept", help="Structure a concept from JSON spec and output TOML")(concept_cmd)
app.command(name="pipe", help="Structure a pipe from JSON spec and output TOML")(pipe_cmd)
-app.command(name="assemble", help="Assemble a complete .mthds bundle from TOML parts")(assemble_cmd)
app.command(name="models", help="List available model presets, aliases, and talent mappings")(agent_models_cmd)
app.command(name="doctor", help="Check Pipelex configuration health and auto-fix issues")(agent_doctor_cmd)
diff --git a/pipelex/cli/agent_cli/commands/agent_output.py b/pipelex/cli/agent_cli/commands/agent_output.py
index 535e43e13..5d00d4798 100644
--- a/pipelex/cli/agent_cli/commands/agent_output.py
+++ b/pipelex/cli/agent_cli/commands/agent_output.py
@@ -8,6 +8,13 @@ import typer
from pipelex.pipeline.validate_bundle import ValidateBundleError
from pipelex.tools.misc.json_utils import clean_json_dumps
+from pipelex.types import StrEnum
+
+
+class CliOutputFormat(StrEnum):
+ JSON = "json"
+ MARKDOWN = "markdown"
+
AGENT_ERROR_HINTS: dict[str, str] = {
# Model/routing errors
@@ -45,9 +52,6 @@ AGENT_ERROR_HINTS: dict[str, str] = {
"PipelineRequestError": "Check that pipe_code or mthds_content is provided",
# Graph errors
"GraphSpecParseError": "Validate graphspec.json structure; ensure it matches the expected GraphSpec schema",
- # Builder/assembler errors
- "ConceptLoadError": "Check TOML syntax in the concept source file or inline string",
- "PipeLoadError": "Check TOML syntax in the pipe source file or inline string",
# Input/type errors
"JsonTypeError": "Input file must be a JSON object {...}, not an array or scalar value",
"BundleError": "Bundle must declare a 'main_pipe' or use the --pipe flag to specify which pipe to run",
@@ -79,8 +83,6 @@ AGENT_ERROR_DOMAINS: dict[str, str] = {
"BundleError": "input",
"PipelineRequestError": "input",
"GraphSpecParseError": "input",
- "ConceptLoadError": "input",
- "PipeLoadError": "input",
"UnknownCommandError": "input",
# config = environment/config changes needed
"ClientAuthenticationError": "config",
diff --git a/pipelex/cli/agent_cli/commands/assemble_cmd.py b/pipelex/cli/agent_cli/commands/assemble_cmd.py
deleted file mode 100644
index fb674318a..000000000
--- a/pipelex/cli/agent_cli/commands/assemble_cmd.py
+++ /dev/null
@@ -1,173 +0,0 @@
-"""Agent CLI assemble command - assemble TOML bundle from parts with JSON output."""
-
-# pyright: reportUnknownMemberType=false
-# pyright: reportArgumentType=false
-# mypy: disable-error-code="arg-type"
-
-from pathlib import Path
-from typing import Annotated
-
-import tomlkit
-import typer
-
-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success
-
-
-def _merge_toml_sections(
- base_doc: tomlkit.TOMLDocument,
- section_name: str,
- content_to_merge: tomlkit.TOMLDocument,
-) -> None:
- """Merge a section from content into the base document.
-
- Args:
- base_doc: The base TOML document to merge into.
- section_name: The name of the section (e.g., "concept", "pipe").
- content_to_merge: The content document to merge (tomlkit preserves inline tables).
- """
- if section_name not in content_to_merge:
- return
-
- section_content = content_to_merge[section_name]
- if not section_content:
- return
-
- if section_name not in base_doc:
- base_doc.add(section_name, tomlkit.table())
-
- existing_section = base_doc[section_name]
- for key, value in section_content.items(): # type: ignore[union-attr]
- existing_section[key] = value # type: ignore[index]
-
-
-def _load_toml_content(source: str) -> tomlkit.TOMLDocument:
- """Load TOML content from either a file path or inline string.
-
- Uses tomlkit to preserve inline tables and formatting.
-
- Args:
- source: Either a file path or inline TOML string.
-
- Returns:
- Parsed TOML document that preserves formatting.
- """
- source_path = Path(source)
- if source_path.exists() and source_path.is_file():
- with open(source_path, encoding="utf-8") as the_file:
- return tomlkit.load(the_file)
- return tomlkit.parse(source)
-
-
-def assemble_cmd(
- domain: Annotated[
- str,
- typer.Option("--domain", "-d", help="Domain code for the bundle (snake_case)"),
- ],
- main_pipe: Annotated[
- str,
- typer.Option("--main-pipe", "-m", help="Main pipe code for the bundle"),
- ],
- output: Annotated[
- str | None,
- typer.Option("--output", "-o", help="Output file path (.mthds). Omit to return TOML in JSON response."),
- ] = None,
- description: Annotated[
- str | None,
- typer.Option("--description", help="Description of the bundle"),
- ] = None,
- system_prompt: Annotated[
- str | None,
- typer.Option("--system-prompt", help="Default system prompt for LLM pipes"),
- ] = None,
- concepts: Annotated[
- list[str] | None,
- typer.Option("--concepts", "-c", help="TOML file(s) or inline TOML containing concept definitions"),
- ] = None,
- pipes: Annotated[
- list[str] | None,
- typer.Option("--pipes", "-p", help="TOML file(s) or inline TOML containing pipe definitions"),
- ] = None,
-) -> None:
- """Assemble a complete .mthds bundle from individual TOML parts.
-
- Combines domain configuration, concepts, and pipes into a single valid
- Pipelex bundle file. Each --concepts and --pipes argument can be either
- a file path or inline TOML content.
-
- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.
-
- Examples:
- pipelex-agent assemble --domain my_domain --main-pipe main
- --concepts concepts.toml --pipes pipes.toml
-
- pipelex-agent assemble --domain my_domain --main-pipe main
- --concepts concepts.toml --pipes pipes.toml --output bundle.mthds
- """
- try:
- # Create base document with domain header
- doc = tomlkit.document()
- doc.add("domain", domain)
-
- if description:
- doc.add("description", description)
-
- if system_prompt:
- doc.add("system_prompt", system_prompt)
-
- doc.add("main_pipe", main_pipe)
-
- # Process concept sources
- if concepts:
- for concept_source in concepts:
- try:
- concept_content = _load_toml_content(concept_source)
- _merge_toml_sections(doc, "concept", concept_content)
- except FileNotFoundError as exc:
- agent_error(f"Concept file not found: {concept_source}", "FileNotFoundError", cause=exc)
- except Exception as exc:
- agent_error(f"Failed to load concepts from '{concept_source}': {exc}", "ConceptLoadError", cause=exc)
-
- # Process pipe sources
- if pipes:
- for pipe_source in pipes:
- try:
- pipe_content = _load_toml_content(pipe_source)
- _merge_toml_sections(doc, "pipe", pipe_content)
- except FileNotFoundError as exc:
- agent_error(f"Pipe file not found: {pipe_source}", "FileNotFoundError", cause=exc)
- except Exception as exc:
- agent_error(f"Failed to load pipes from '{pipe_source}': {exc}", "PipeLoadError", cause=exc)
-
- toml_content = tomlkit.dumps(doc)
- if not toml_content.endswith("\n"):
- toml_content += "\n"
-
- if output is None:
- # Stdout mode (default): return TOML in JSON response
- agent_success(
- {
- "success": True,
- "toml": toml_content,
- "domain": domain,
- "main_pipe": main_pipe,
- }
- )
- else:
- # File mode: write to disk
- output_path = Path(output)
- output_path.parent.mkdir(parents=True, exist_ok=True)
- Path(output_path).write_text(toml_content, encoding="utf-8")
- agent_success(
- {
- "success": True,
- "bundle_path": str(output_path.resolve()),
- "domain": domain,
- "main_pipe": main_pipe,
- }
- )
-
- except typer.Exit:
- raise
-
- except Exception as exc:
- agent_error(str(exc), type(exc).__name__, cause=exc)
diff --git a/pipelex/cli/agent_cli/commands/concept_cmd.py b/pipelex/cli/agent_cli/commands/concept_cmd.py
index 5b51c3245..dd64700e0 100644
--- a/pipelex/cli/agent_cli/commands/concept_cmd.py
+++ b/pipelex/cli/agent_cli/commands/concept_cmd.py
@@ -1,4 +1,4 @@
-"""Agent CLI concept command - structure concepts from JSON specs with JSON/TOML output."""
+"""Agent CLI concept command - structure concepts from JSON specs with raw TOML output."""
# pyright: reportUnknownMemberType=false
@@ -10,7 +10,7 @@ import typer
from pydantic import ValidationError
from pipelex.builder.concept.concept_spec import ConceptSpec, ConceptStructureSpec
-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success
+from pipelex.cli.agent_cli.commands.agent_output import agent_error
from pipelex.language.toml_string_utils import format_toml_string
from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error_for_agent
@@ -146,7 +146,7 @@ def concept_cmd(
Takes a concept specification in JSON format and converts it to valid Pipelex
TOML format. Validates the spec before conversion.
- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.
+ Outputs raw TOML to stdout on success, JSON to stderr on error with exit code 1.
JSON spec format:
{
@@ -192,13 +192,7 @@ def concept_cmd(
concept_spec = _parse_concept_spec_from_json(spec_data)
toml_content = _concept_spec_to_toml(concept_spec)
- agent_success(
- {
- "success": True,
- "concept_code": concept_spec.the_concept_code,
- "toml": toml_content,
- }
- )
+ print(toml_content, end="" if toml_content.endswith("\n") else "\n")
except ValidationError as exc:
message, details = format_pydantic_validation_error_for_agent(exc)
diff --git a/pipelex/cli/agent_cli/commands/doctor_cmd.py b/pipelex/cli/agent_cli/commands/doctor_cmd.py
index a8f1290d2..ca488d184 100644
--- a/pipelex/cli/agent_cli/commands/doctor_cmd.py
+++ b/pipelex/cli/agent_cli/commands/doctor_cmd.py
@@ -1,10 +1,10 @@
-"""Agent CLI doctor command -- JSON health report with no interactive prompts."""
+"""Agent CLI doctor command -- health report with no interactive prompts."""
from typing import Annotated, Any
import typer
-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success
+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, agent_success
from pipelex.cli.commands.doctor_cmd import (
ConfigLocationInfo,
check_backend_credentials,
@@ -16,6 +16,77 @@ from pipelex.cli.commands.doctor_cmd import (
from pipelex.system.configuration.config_loader import config_manager
+def _status_icon(healthy: bool) -> str:
+ """Return a status emoji: checkmark for healthy, warning for unhealthy."""
+ return "\u2705" if healthy else "\u26a0\ufe0f"
+
+
+def _format_doctor_markdown(result: dict[str, Any]) -> str:
+ """Format the doctor result dict as markdown with bullet points."""
+ all_healthy: bool = result["all_healthy"]
+ config_location: dict[str, Any] = result["config_location"]
+ checks: dict[str, Any] = result["checks"]
+
+ status_text = f"All healthy {_status_icon(True)}" if all_healthy else f"Issues found {_status_icon(False)}"
+ location_type = "project-local" if config_location.get("is_project_local") else "global"
+
+ lines: list[str] = [
+ "# Pipelex Health Check",
+ "",
+ f"**Status:** {status_text}",
+ f"**Config location:** {config_location['config_dir']} ({location_type})",
+ ]
+
+ # Config Files
+ config_check = checks["config_files"]
+ lines.append(f"\n## Config Files \u2014 {_status_icon(config_check['healthy'])}\n")
+ lines.append(config_check["message"])
+
+ # Telemetry
+ telemetry_check = checks["telemetry"]
+ lines.append(f"\n## Telemetry \u2014 {_status_icon(telemetry_check['healthy'])}\n")
+ lines.append(telemetry_check["message"])
+
+ # Backend Credentials
+ creds_check = checks["backend_credentials"]
+ lines.append(f"\n## Backend Credentials \u2014 {_status_icon(creds_check['healthy'])}\n")
+ lines.append(creds_check["message"])
+ for backend_entry in creds_check.get("backends", []):
+ name = backend_entry["backend_name"]
+ if backend_entry["all_credentials_valid"]:
+ lines.append(f"- **{name}**: All credentials valid")
+ else:
+ issues: list[str] = []
+ missing = backend_entry.get("missing_vars")
+ if missing:
+ issues.append(f"missing: {', '.join(missing)}")
+ placeholder = backend_entry.get("placeholder_vars")
+ if placeholder:
+ issues.append(f"placeholder: {', '.join(placeholder)}")
+ lines.append(f"- **{name}**: {'; '.join(issues) or 'credentials invalid'}")
+
+ # Models
+ models_check = checks["models"]
+ lines.append(f"\n## Models \u2014 {_status_icon(models_check['healthy'])}\n")
+ lines.append(models_check["message"])
+ for file_entry in models_check.get("backend_files", []):
+ name = file_entry["backend_name"]
+ if file_entry["is_valid"]:
+ lines.append(f"- **{name}** (`{file_entry['file_path']}`): Valid")
+ else:
+ error_msg = file_entry.get("error_message", "unknown error")
+ lines.append(f"- **{name}** (`{file_entry['file_path']}`): Invalid \u2014 {error_msg}")
+
+ # Recommended Actions
+ recommended = result.get("recommended_actions", [])
+ if recommended:
+ lines.append("\n## Recommended Actions\n")
+ for index, action in enumerate(recommended, 1):
+ lines.append(f"{index}. {action}")
+
+ return "\n".join(lines)
+
+
def agent_doctor_cmd(
global_: Annotated[
bool,
@@ -25,14 +96,19 @@ def agent_doctor_cmd(
help="Force checking the global ~/.pipelex/ directory.",
),
] = False,
+ output_format: Annotated[
+ CliOutputFormat,
+ typer.Option("--format", help="Output format: markdown (default) or json (structured)"),
+ ] = CliOutputFormat.MARKDOWN,
) -> None:
- """Check Pipelex configuration health and output a JSON report.
+ """Check Pipelex configuration health and output a diagnostic report.
Unlike the human CLI doctor, this command:
- - Outputs structured JSON to stdout (never Rich panels or colors)
- Does not offer interactive fixes (diagnostic-only)
- Includes ``recommended_actions`` for programmatic remediation
+ Default output is markdown; use --format json for structured JSON.
+
Target directory: auto-detects project .pipelex/ if present, else ~/.pipelex/.
Use --global/-g to force checking the global ~/.pipelex/ directory.
"""
@@ -143,4 +219,8 @@ def agent_doctor_cmd(
if recommended_actions:
result["recommended_actions"] = recommended_actions
- agent_success(result)
+ match output_format:
+ case CliOutputFormat.JSON:
+ agent_success(result)
+ case CliOutputFormat.MARKDOWN:
+ print(_format_doctor_markdown(result))
diff --git a/pipelex/cli/agent_cli/commands/models_cmd.py b/pipelex/cli/agent_cli/commands/models_cmd.py
index d61d14803..2164d3157 100644
--- a/pipelex/cli/agent_cli/commands/models_cmd.py
+++ b/pipelex/cli/agent_cli/commands/models_cmd.py
@@ -5,7 +5,7 @@ from typing import Annotated, Any
import typer
from pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_cli
-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success
+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, agent_success
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.cogt.model_backends.model_type import ModelType
from pipelex.cogt.models.model_deck import ModelDeck
@@ -223,6 +223,66 @@ def _build_talent_mappings_for_category(
return _filter_talent_mappings_by_backend(mappings, presets_dict, model_deck, model_type, backend)
+CATEGORY_DISPLAY_NAMES: dict[str, str] = {
+ "llm": "LLM",
+ "img_gen": "Image Generation",
+ "extract": "Extract",
+ "search": "Search",
+}
+
+
+def _format_models_markdown(result: dict[str, Any]) -> str:
+ """Format the models result dict as markdown with bullet points."""
+ lines: list[str] = ["# Available Models"]
+
+ presets: dict[str, list[dict[str, Any]]] = result["presets"]
+ aliases: dict[str, dict[str, str]] = result["aliases"]
+ waterfalls: dict[str, dict[str, list[str]]] = result["waterfalls"]
+ talent_mappings: dict[str, dict[str, str]] = result["talent_mappings"]
+
+ for category_key, display_name in CATEGORY_DISPLAY_NAMES.items():
+ cat_presets = presets.get(category_key, [])
+ cat_aliases = aliases.get(category_key, {})
+ cat_waterfalls = waterfalls.get(category_key, {})
+ cat_talents = talent_mappings.get(category_key, {})
+
+ if not cat_presets and not cat_aliases and not cat_waterfalls and not cat_talents:
+ continue
+
+ lines.append(f"\n## {display_name}")
+
+ if cat_presets:
+ lines.append("\n### Presets\n")
+ for preset in cat_presets:
+ description = preset.get("description", "")
+ if description:
+ lines.append(f"- **{preset['name']}**: {description}")
+ else:
+ lines.append(f"- **{preset['name']}**")
+
+ if cat_aliases:
+ lines.append("\n### Aliases\n")
+ for alias_name, alias_target in cat_aliases.items():
+ lines.append(f"- **{alias_name}** \u2192 {alias_target}")
+
+ if cat_waterfalls:
+ lines.append("\n### Waterfalls\n")
+ for waterfall_name, fallback_list in cat_waterfalls.items():
+ chain = " \u2192 ".join(fallback_list)
+ lines.append(f"- **{waterfall_name}**: {chain}")
+
+ if cat_talents:
+ lines.append("\n### Talent Mappings\n")
+ for talent_name, preset_name in cat_talents.items():
+ lines.append(f"- **{talent_name}** \u2192 {preset_name}")
+
+ hint = result.get("talent_mappings_usage_hint")
+ if hint:
+ lines.append(f"\n---\n\n{hint}")
+
+ return "\n".join(lines)
+
+
def agent_models_cmd(
ctx: typer.Context,
model_type: Annotated[
@@ -233,11 +293,15 @@ def agent_models_cmd(
str | None,
typer.Option("--backend", "-b", help="Filter by backend name"),
] = None,
+ output_format: Annotated[
+ CliOutputFormat,
+ typer.Option("--format", help="Output format: markdown (default) or json (structured)"),
+ ] = CliOutputFormat.MARKDOWN,
) -> None:
"""List available model presets, aliases, waterfalls, and talent mappings.
- Outputs structured JSON to stdout with all model configuration
- that an agent needs to reference when building pipelines.
+ Outputs model configuration that an agent needs to reference when building pipelines.
+ Default output is markdown; use --format json for structured JSON.
"""
try:
make_pipelex_for_agent_cli(log_level=ctx.obj["log_level"], needs_inference=False, needs_model_specs=backend is not None)
@@ -287,7 +351,11 @@ def agent_models_cmd(
),
}
- agent_success(result)
+ match output_format:
+ case CliOutputFormat.JSON:
+ agent_success(result)
+ case CliOutputFormat.MARKDOWN:
+ print(_format_models_markdown(result))
except SystemExit:
# agent_error already handled and called sys.exit
raise
diff --git a/pipelex/cli/agent_cli/commands/pipe_cmd.py b/pipelex/cli/agent_cli/commands/pipe_cmd.py
index 2fa6db90f..c89599372 100644
--- a/pipelex/cli/agent_cli/commands/pipe_cmd.py
+++ b/pipelex/cli/agent_cli/commands/pipe_cmd.py
@@ -1,4 +1,4 @@
-"""Agent CLI pipe command - structure pipes from JSON specs with JSON/TOML output."""
+"""Agent CLI pipe command - structure pipes from JSON specs with raw TOML output."""
# pyright: reportUnknownMemberType=false
# pyright: reportArgumentType=false
@@ -27,7 +27,7 @@ from pipelex.builder.pipe.pipe_spec_map import pipe_type_to_spec_class
from pipelex.builder.talents.extract_talent import ExtractTalent
from pipelex.builder.talents.img_gen_talent import ImgGenTalent
from pipelex.builder.talents.llm_talent import LLMTalent
-from pipelex.cli.agent_cli.commands.agent_output import agent_error, agent_success
+from pipelex.cli.agent_cli.commands.agent_output import agent_error
from pipelex.core.pipes.pipe_blueprint import PipeType
from pipelex.language.toml_string_utils import format_toml_string
from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error_for_agent
@@ -337,7 +337,7 @@ def pipe_cmd(
Takes a pipe specification in JSON format and converts it to valid Pipelex
TOML format. Validates the spec before conversion.
- Outputs JSON to stdout on success, JSON to stderr on error with exit code 1.
+ Outputs raw TOML to stdout on success, JSON to stderr on error with exit code 1.
JSON spec format (varies by pipe type):
@@ -407,14 +407,7 @@ def pipe_cmd(
pipe_spec = _parse_pipe_spec_from_json(resolved_pipe_type, spec_data)
toml_content = _pipe_spec_to_toml(pipe_spec)
- agent_success(
- {
- "success": True,
- "pipe_code": pipe_spec.pipe_code,
- "pipe_type": resolved_pipe_type,
- "toml": toml_content,
- }
- )
+ print(toml_content, end="" if toml_content.endswith("\n") else "\n")
except ValidationError as exc:
message, details = format_pydantic_validation_error_for_agent(exc)
diff --git a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
index 87df5abf5..f57bcf80e 100644
--- a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
+++ b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
@@ -11,6 +11,7 @@ import typer
if TYPE_CHECKING:
from pytest_mock import MockerFixture
+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat
from pipelex.cli.agent_cli.commands.doctor_cmd import agent_doctor_cmd
from pipelex.cogt.model_backends.backend_library import BackendCredentialsReport
@@ -37,7 +38,7 @@ class TestAgentDoctorCmd:
return_value=(True, "Models valid", {}),
)
- agent_doctor_cmd()
+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert parsed["success"] is True
@@ -63,7 +64,7 @@ class TestAgentDoctorCmd:
return_value=(True, "Models valid", {}),
)
- agent_doctor_cmd()
+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert parsed["all_healthy"] is False
@@ -101,7 +102,7 @@ class TestAgentDoctorCmd:
return_value=(True, "Models valid", {}),
)
- agent_doctor_cmd()
+ agent_doctor_cmd(output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert parsed["all_healthy"] is False
diff --git a/tests/unit/pipelex/cli/test_agent_models_cmd.py b/tests/unit/pipelex/cli/test_agent_models_cmd.py
index b80f2d4a0..56b9c04c8 100644
--- a/tests/unit/pipelex/cli/test_agent_models_cmd.py
+++ b/tests/unit/pipelex/cli/test_agent_models_cmd.py
@@ -9,6 +9,7 @@ if TYPE_CHECKING:
import pytest
from pytest_mock import MockerFixture
+from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat
from pipelex.cli.agent_cli.commands.models_cmd import ModelCategory, agent_models_cmd
from pipelex.cogt.model_backends.model_type import ModelType
@@ -151,7 +152,7 @@ class TestAgentModelsCmd:
"""No filters should return all 3 categories in every section."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx)
+ agent_models_cmd(ctx=agent_ctx, output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert parsed["success"] is True
@@ -164,7 +165,7 @@ class TestAgentModelsCmd:
"""No filters should include all preset entries with correct data."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx)
+ agent_models_cmd(ctx=agent_ctx, output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
llm_preset_names = [preset["name"] for preset in parsed["presets"]["llm"]]
@@ -178,7 +179,7 @@ class TestAgentModelsCmd:
"""--type llm should return only llm keys in all sections."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM])
+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
for section in ("presets", "aliases", "waterfalls", "talent_mappings"):
@@ -190,7 +191,7 @@ class TestAgentModelsCmd:
"""--type extract should return only extract keys in all sections."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.EXTRACT])
+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.EXTRACT], output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
for section in ("presets", "aliases", "waterfalls", "talent_mappings"):
@@ -202,7 +203,7 @@ class TestAgentModelsCmd:
"""--type img_gen should return only img_gen keys in all sections."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.IMG_GEN])
+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.IMG_GEN], output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
for section in ("presets", "aliases", "waterfalls", "talent_mappings"):
@@ -214,7 +215,7 @@ class TestAgentModelsCmd:
"""--type llm --type img_gen should return both llm and img_gen, but not extract."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM, ModelCategory.IMG_GEN])
+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM, ModelCategory.IMG_GEN], output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
for section in ("presets", "aliases", "waterfalls", "talent_mappings"):
@@ -226,7 +227,7 @@ class TestAgentModelsCmd:
"""--backend openai should filter presets/aliases/waterfalls to only openai-backed models."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, backend="openai")
+ agent_models_cmd(ctx=agent_ctx, backend="openai", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
# LLM: only 'fast' preset (gpt-4o-mini on openai), not 'smart' (claude-sonnet on anthropic)
@@ -245,7 +246,7 @@ class TestAgentModelsCmd:
"""--backend anthropic should include only anthropic-backed models."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, backend="anthropic")
+ agent_models_cmd(ctx=agent_ctx, backend="anthropic", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
llm_preset_names = [preset["name"] for preset in parsed["presets"]["llm"]]
@@ -258,7 +259,7 @@ class TestAgentModelsCmd:
"""--type llm --backend openai should combine both filters."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], backend="openai")
+ agent_models_cmd(ctx=agent_ctx, model_type=[ModelCategory.LLM], backend="openai", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
# Only llm category
@@ -274,7 +275,7 @@ class TestAgentModelsCmd:
"""--backend nonexistent should produce empty presets/aliases/waterfalls/talent_mappings."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, backend="nonexistent")
+ agent_models_cmd(ctx=agent_ctx, backend="nonexistent", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert parsed["success"] is True
@@ -286,7 +287,7 @@ class TestAgentModelsCmd:
"""--backend anthropic should include llm waterfall (has claude-sonnet) but not img waterfall."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, backend="anthropic")
+ agent_models_cmd(ctx=agent_ctx, backend="anthropic", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert "llm-wf" in parsed["waterfalls"]["llm"]
@@ -297,7 +298,7 @@ class TestAgentModelsCmd:
"""--backend openai should include LLM talent_mapping (fast->gpt-4o-mini) and img_gen, but not extract."""
_setup_mocks(mocker)
- agent_models_cmd(ctx=agent_ctx, backend="openai")
+ agent_models_cmd(ctx=agent_ctx, backend="openai", output_format=CliOutputFormat.JSON)
parsed = json.loads(capsys.readouterr().out)
assert "text_gen" in parsed["talent_mappings"]["llm"]
diff --git a/tests/unit/pipelex/cli/test_format_doctor_markdown.py b/tests/unit/pipelex/cli/test_format_doctor_markdown.py
new file mode 100644
index 000000000..f89c390ec
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_format_doctor_markdown.py
@@ -0,0 +1,142 @@
+"""Unit tests for the doctor command markdown formatter."""
+
+from __future__ import annotations
+
+from typing import Any, ClassVar
+
+from pipelex.cli.agent_cli.commands.doctor_cmd import (
+ _format_doctor_markdown, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+)
+
+
+class TestFormatDoctorMarkdown:
+ """Tests for _format_doctor_markdown output."""
+
+ _HEALTHY_RESULT: ClassVar[dict[str, Any]] = {
+ "success": True,
+ "all_healthy": True,
+ "config_location": {
+ "config_dir": "/home/user/.pipelex",
+ "is_project_local": False,
+ "project_root": None,
+ "global_config_dir": "/home/user/.pipelex",
+ },
+ "checks": {
+ "config_files": {"healthy": True, "message": "All config files present", "missing_count": 0},
+ "telemetry": {"healthy": True, "message": "Telemetry configured"},
+ "backend_credentials": {"healthy": True, "message": "All backends healthy", "backends": []},
+ "models": {"healthy": True, "message": "Models valid", "backend_files": []},
+ },
+ }
+
+ _UNHEALTHY_RESULT: ClassVar[dict[str, Any]] = {
+ "success": True,
+ "all_healthy": False,
+ "config_location": {
+ "config_dir": "/project/.pipelex",
+ "is_project_local": True,
+ "project_root": "/project",
+ "global_config_dir": "/home/user/.pipelex",
+ },
+ "checks": {
+ "config_files": {"healthy": True, "message": "OK", "missing_count": 0},
+ "telemetry": {"healthy": False, "message": "Config format outdated"},
+ "backend_credentials": {
+ "healthy": False,
+ "message": "1 backend has issues",
+ "backends": [
+ {"backend_name": "openai", "all_credentials_valid": True},
+ {"backend_name": "anthropic", "all_credentials_valid": False, "missing_vars": ["ANTHROPIC_API_KEY"]},
+ ],
+ },
+ "models": {
+ "healthy": True,
+ "message": "Models valid",
+ "backend_files": [
+ {"backend_name": "openai", "file_path": ".pipelex/inference/backends/openai.toml", "is_valid": True},
+ ],
+ },
+ },
+ "recommended_actions": [
+ "Set environment variable: ANTHROPIC_API_KEY",
+ ],
+ }
+
+ def test_healthy_output_contains_status_and_checkmark(self) -> None:
+ """Healthy result should show 'All healthy' with checkmark emoji."""
+ output = _format_doctor_markdown(self._HEALTHY_RESULT)
+ assert "All healthy \u2705" in output
+
+ def test_healthy_output_shows_global_location(self) -> None:
+ """Global config should be labeled as 'global'."""
+ output = _format_doctor_markdown(self._HEALTHY_RESULT)
+ assert "/home/user/.pipelex (global)" in output
+
+ def test_healthy_output_has_no_recommended_actions(self) -> None:
+ """Healthy result should not include a Recommended Actions section."""
+ output = _format_doctor_markdown(self._HEALTHY_RESULT)
+ assert "Recommended Actions" not in output
+
+ def test_unhealthy_output_shows_warning_status(self) -> None:
+ """Unhealthy result should show 'Issues found' with warning emoji."""
+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)
+ assert "Issues found \u26a0\ufe0f" in output
+
+ def test_unhealthy_output_shows_project_local(self) -> None:
+ """Project-local config should be labeled as 'project-local'."""
+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)
+ assert "/project/.pipelex (project-local)" in output
+
+ def test_unhealthy_output_lists_missing_credentials(self) -> None:
+ """Missing credentials should appear in the backend credentials section."""
+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)
+ assert "ANTHROPIC_API_KEY" in output
+ assert "**openai**: All credentials valid" in output
+
+ def test_unhealthy_output_includes_recommended_actions(self) -> None:
+ """Recommended actions should appear as a numbered list."""
+ output = _format_doctor_markdown(self._UNHEALTHY_RESULT)
+ assert "1. Set environment variable: ANTHROPIC_API_KEY" in output
+
+ def test_invalid_model_file_shows_error(self) -> None:
+ """Invalid model file should show error message in output."""
+ result: dict[str, Any] = {
+ **self._HEALTHY_RESULT,
+ "all_healthy": False,
+ "checks": {
+ **self._HEALTHY_RESULT["checks"],
+ "models": {
+ "healthy": False,
+ "message": "Issues found",
+ "backend_files": [
+ {
+ "backend_name": "bad_backend",
+ "file_path": ".pipelex/inference/backends/bad.toml",
+ "is_valid": False,
+ "error_message": "Parse error on line 5",
+ },
+ ],
+ },
+ },
+ }
+ output = _format_doctor_markdown(result)
+ assert "Invalid \u2014 Parse error on line 5" in output
+
+ def test_credentials_invalid_without_details_shows_fallback(self) -> None:
+ """Backend with invalid credentials but no missing/placeholder vars shows fallback text."""
+ result: dict[str, Any] = {
+ **self._HEALTHY_RESULT,
+ "all_healthy": False,
+ "checks": {
+ **self._HEALTHY_RESULT["checks"],
+ "backend_credentials": {
+ "healthy": False,
+ "message": "Issues",
+ "backends": [
+ {"backend_name": "mystery", "all_credentials_valid": False},
+ ],
+ },
+ },
+ }
+ output = _format_doctor_markdown(result)
+ assert "**mystery**: credentials invalid" in output
diff --git a/tests/unit/pipelex/cli/test_format_models_markdown.py b/tests/unit/pipelex/cli/test_format_models_markdown.py
new file mode 100644
index 000000000..1fa9365a0
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_format_models_markdown.py
@@ -0,0 +1,125 @@
+"""Unit tests for the models command markdown formatter."""
+
+from __future__ import annotations
+
+from typing import Any, ClassVar
+
+from pipelex.cli.agent_cli.commands.models_cmd import (
+ _format_models_markdown, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+)
+
+
+class TestFormatModelsMarkdown:
+ """Tests for _format_models_markdown output."""
+
+ _FULL_RESULT: ClassVar[dict[str, Any]] = {
+ "success": True,
+ "presets": {
+ "llm": [
+ {"name": "writing-creative", "description": "Creative writing"},
+ {"name": "engineering-code"},
+ ],
+ "img_gen": [{"name": "gen-image", "description": "Image generation"}],
+ "extract": [],
+ "search": [],
+ },
+ "aliases": {
+ "llm": {"fast": "gpt4o-mini", "smart": "claude-sonnet"},
+ "img_gen": {},
+ "extract": {},
+ "search": {},
+ },
+ "waterfalls": {
+ "llm": {"robust": ["gpt4o", "claude-sonnet", "gemini"]},
+ "img_gen": {},
+ "extract": {},
+ "search": {},
+ },
+ "talent_mappings": {
+ "llm": {"creative-writer": "writing-creative"},
+ "img_gen": {},
+ "extract": {},
+ "search": {},
+ },
+ "talent_mappings_usage_hint": "Use the talent name as the value for llm_talent",
+ }
+
+ def test_output_starts_with_heading(self) -> None:
+ """Output should start with the main heading."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert output.startswith("# Available Models")
+
+ def test_llm_category_present(self) -> None:
+ """LLM category with data should appear as a section."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "## LLM" in output
+
+ def test_preset_with_description(self) -> None:
+ """Preset with description should show name and description."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "- **writing-creative**: Creative writing" in output
+
+ def test_preset_without_description(self) -> None:
+ """Preset without description should show only the name."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "- **engineering-code**" in output
+
+ def test_aliases_use_arrow(self) -> None:
+ """Aliases should use arrow notation."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "- **fast** \u2192 gpt4o-mini" in output
+
+ def test_waterfalls_show_chain(self) -> None:
+ """Waterfalls should show the full fallback chain with arrows."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "gpt4o \u2192 claude-sonnet \u2192 gemini" in output
+
+ def test_talent_mappings_present(self) -> None:
+ """Talent mappings should use arrow notation."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "- **creative-writer** \u2192 writing-creative" in output
+
+ def test_hint_present(self) -> None:
+ """Usage hint should appear at the bottom."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "Use the talent name as the value for llm_talent" in output
+
+ def test_empty_categories_omitted(self) -> None:
+ """Categories with no data should not appear in output."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "## Extract" not in output
+ assert "## Search" not in output
+
+ def test_img_gen_category_present(self) -> None:
+ """Image Generation category with presets should appear."""
+ output = _format_models_markdown(self._FULL_RESULT)
+ assert "## Image Generation" in output
+ assert "- **gen-image**: Image generation" in output
+
+ def test_all_empty_produces_heading_only(self) -> None:
+ """Result with all empty sections should produce only the heading and hint."""
+ empty_result: dict[str, Any] = {
+ "presets": {"llm": [], "img_gen": [], "extract": [], "search": []},
+ "aliases": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},
+ "waterfalls": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},
+ "talent_mappings": {"llm": {}, "img_gen": {}, "extract": {}, "search": {}},
+ "talent_mappings_usage_hint": "hint text",
+ }
+ output = _format_models_markdown(empty_result)
+ assert "# Available Models" in output
+ assert "## LLM" not in output
+ assert "hint text" in output
+
+ def test_empty_subsections_omitted_within_category(self) -> None:
+ """Within a category, subsections with no data should be omitted."""
+ result: dict[str, Any] = {
+ "presets": {"llm": [{"name": "only-preset"}]},
+ "aliases": {"llm": {}},
+ "waterfalls": {"llm": {}},
+ "talent_mappings": {"llm": {}},
+ }
+ output = _format_models_markdown(result)
+ assert "### Presets" in output
+ assert "### Aliases" not in output
+ assert "### Waterfalls" not in output
+ assert "### Talent Mappings" not in output