Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Openrouter-tooling · L-261020-debfa7 (#1472)
Louis Choquel54c5577fa456
diff --git a/.claude/skills/add-model/SKILL.md b/.claude/skills/add-model/SKILL.mdindex 2758d79c5..4cb3b006f 100644--- a/.claude/skills/add-model/SKILL.md+++ b/.claude/skills/add-model/SKILL.md@@ -20,7 +20,7 @@ The hosted model catalog is not in this repository: the hosted plane carries it.## 1. Establish the facts, from the provider-Build a fact sheet before touching a file, and give every fact its source. The provider's own documentation is the primary source: its model page, its pricing page and its API reference name the exact model id, the input and output modalities, the prices, whether it reasons and how that is controlled, and which request parameters it refuses. Fetch those pages. OpenRouter is a useful cross-check for prices and modalities (`references/openrouter-price-lookup.md`), not a substitute: it lags new releases, and it cannot tell you which parameters a model deprecates.+Build a fact sheet before touching a file, and give every fact its source. The provider's own documentation is the primary source: its model page, its pricing page and its API reference name the exact model id, the input and output modalities, the prices, whether it reasons and how that is controlled, and which request parameters it refuses. Fetch those pages. Use `.venv/bin/pipelex-dev openrouter list-models -f <name> --json` from a development checkout for OpenRouter facts; read `ok` and its `models` list. OpenRouter is a useful cross-check for prices and modalities (`references/openrouter-price-lookup.md`), not a substitute: it lags new releases, and it cannot tell you which parameters a model deprecates.| Fact | Where it goes | Watch for ||---|---|---|diff --git a/.claude/skills/add-model/references/openrouter-price-lookup.md b/.claude/skills/add-model/references/openrouter-price-lookup.mdindex 834e28a2e..580f9db02 100644--- a/.claude/skills/add-model/references/openrouter-price-lookup.md+++ b/.claude/skills/add-model/references/openrouter-price-lookup.md@@ -1,91 +1,9 @@-# OpenRouter Price Lookup — API Reference+# OpenRouter model research-Use this reference to look up model costs when the user doesn't have them handy.-OpenRouter aggregates pricing across providers, making it a convenient single-source for input/output token costs.+Use `.venv/bin/pipelex-dev openrouter list-models -f <name> --json` from a Pipelex development checkout provisioned with `make install`. Read `ok` and the `models` list in its JSON envelope. `--save` writes disabled draft backend TOML under `generated/`, refuses overwrites and reports the path in `saved_to`.-## Quick lookup via WebFetch+The command reads `https://openrouter.ai/api/v1/models?output_modalities=all`, including image models, without the old frontend API or remote-config checkout. Authentication is optional; it uses `OPENROUTER_API_KEY` when supplied. If the development CLI is unavailable, fetch the same public endpoint directly and filter its `data` list by model ID or name.-The fastest way to get a model's pricing is to fetch from the OpenRouter API and-filter by name. No API key is required for basic lookups.+Prices are per token in the response and per million tokens in Pipelex backend files. Missing prices stay unknown. Context length is distinct from the output-token limit in `top_provider.max_completion_tokens`. Explicit `structured_outputs` metadata is evidence of structured-output support; `tools` alone is not. Consult the provider's own documentation for capabilities, reasoning settings, restrictions and current prices before enabling a generated entry.-### LLM models--```-GET https://openrouter.ai/api/v1/models-```--Response: `{"data": [<model>, ...]}` where each model has:--```json-{- "id": "anthropic/claude-sonnet-4",- "name": "Claude Sonnet 4",- "pricing": {- "prompt": "0.000003",- "completion": "0.000015"- },- "architecture": {- "input_modalities": ["text", "image", "file"],- "output_modalities": ["text"]- },- "supported_parameters": ["tools", "temperature", "top_p", ...]-}-```--### Image generation models--```-GET https://openrouter.ai/api/frontend/models?category=image-generation-```--Response uses a different schema with flat fields:--```json-{- "slug": "openai/gpt-image-1",- "name": "GPT Image 1",- "per_input_token": 0.00001,- "per_output_token": 0.00004,- "input_modalities": ["text", "image"],- "output_modalities": ["image"]-}-```--## Converting prices to our format--OpenRouter prices are **per token**. Our backend TOMLs use **per million tokens**.--```-cost_per_million = float(openrouter_price) * 1_000_000-```--Example: `"prompt": "0.000003"` -> `input = 3.0` in our TOML.--For image gen models from the frontend API, the prices are already floats (not-strings): `per_input_token: 0.00001` -> `input = 10`.--## Modality mapping--OpenRouter uses different names for modalities than we do:--| OpenRouter input | Our input |-|-----------------|-----------|-| `text` | `text` |-| `image` | `images` |-| `file` | `pdf` |--| OpenRouter output | Our output |-|------------------|-----------|-| `text` | `text` |-| `image` | `image` |--If `"tools"` is in `supported_parameters`, the model supports structured output--> add `"structured"` to outputs in our TOML.--## Filtering for a specific model--The API returns all models at once. Filter client-side by matching on `id` (LLM-models) or `slug` (image-generation models), or `name` for either. Both `id` and-`slug` are provider-scoped (e.g., `openai/gpt-5.4`,-`anthropic/claude-4.6-opus`).+The full command and export contract is in `docs/contribute/openrouter-model-research.md`.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a6f319fa5..b08dd1a24 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,8 @@### Added+- **OpenRouter model research**: `pipelex-dev openrouter list-models` looks up the public catalog across modalities, filters and sorts models, and emits Markdown or clean JSON. `--save` writes disabled draft backend TOML with output-token limits and known token prices for review, independently of remote-config tooling.+- **The validation report names the files each step reads**: `PipelexValidationReport`, and so the valid verdict of `/v1/validate`, carries `tree_files_read`, mapping each validated pipe's `pipe_ref` to the files of the method's tree it reads, each a `TreeFileRead` with its `tree_path` and its `use` (`template` for a `PipeDocGen` step's `template_file`, `template_asset` for a file its HTML template names by a literal relative URL, `python_module` for the module holding a `PipeFunc` step's function), an empty list for a pipe reading none, and `tree_file_readers`, mapping each of those files to the pipes that read it. `pipelex-agent validate bundle` and `validate method` carry both in their envelope, and the Markdown of a valid verdict lists them in a "Files the steps read" section.- **A fetched package is checked against the calling method's `methods.lock`**: when a method carries a `methods.lock` at the root of its tree, a package it calls by address that the runtime fetched itself, for this load or by an earlier fetch whose installed copy records its provenance, must have the tree digest the lock records for its address, whatever version the lock names, else the load is refused with the core code `security.integrity_mismatch` (domain `method`, a `422` with its security event on the API server), whose message names the locked and fetched versions and digests and says to pin the reference to the locked version or to lock the method again, or, when an installed copy answered a reference pinning another tag, says so and to remove that copy. A lock not naming the address, or no lock, checks nothing, and packages shipped with the method or installed by another tool are not checked. The lock is read only when a dependency resolves to such a copy, and one the runtime cannot read, one recording a `hash` from before tree digests included, then refuses the load with `method.package_invalid` and the new reason `lock_invalid`.diff --git a/docs/contribute/openrouter-model-research.md b/docs/contribute/openrouter-model-research.mdnew file mode 100644index 000000000..5eee71534--- /dev/null+++ b/docs/contribute/openrouter-model-research.md@@ -0,0 +1,28 @@+# Research OpenRouter models++From a Pipelex development checkout, run `make install` to provision `.venv`, then use its development CLI:++```bash+.venv/bin/pipelex-dev openrouter list-models -f claude+.venv/bin/pipelex-dev openrouter list-models -f claude --json+.venv/bin/pipelex-dev openrouter list-models --sort context_length+.venv/bin/pipelex-dev openrouter list-models -f flux --save+```++It needs no hosted credentials or Pipelex boot. `OPENROUTER_API_KEY` is optional: when present, it is sent as a bearer token; otherwise the catalog request is anonymous. No inference calls are made.++The command reads the [public models API](https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties) with `output_modalities=all`, including image models, and follows catalog pagination. It does not use the frontend API. `--category` forwards OpenRouter's category filter. `--filter` matches a substring of the name or ID, ignoring case. `--sort` accepts `name`, `created` (newest first) and `context_length` (largest **context window** first); the table explicitly calls that column context tokens.++Markdown is the default. `--json` emits an envelope with `ok`, `models`, `saved_to` and `skipped_model_ids`, without a banner. Every model retains the catalog's supplied metadata. Expected failures emit `ok: false` and `error` on stderr in JSON mode, or Markdown on stderr otherwise. An empty lookup is a successful result with no models; attempting to save it is refused.++`--save` writes `generated/openrouter_models.toml` relative to the current directory. Category and name filters add sanitized suffixes, such as `generated/openrouter_models_claude.toml`. Existing files are never overwritten. JSON mode can be combined with saving; `saved_to` reports the generated path.++Exports are disabled draft entries. Review each entry before changing `enabled` to true. The exporter uses the current runtime's backend document validator, quotes model names safely and converts known token prices into USD per million tokens. Missing, negative or nonfinite prices remain absent; absent costs do not imply free inference. Non-token charges, image pricing, routing, reasoning configuration and provider restrictions require manual review. Context length is never exported as `max_tokens`: that field comes only from `top_provider.max_completion_tokens`. Tool support alone never implies structured output; only explicit `structured_outputs` metadata adds that capability. Export includes only models whose outputs are text or images. Unsupported output modalities are skipped and explicitly listed in `skipped_model_ids` in JSON or below the Markdown result; all matched models remain in the lookup. If no matched model is exportable, saving is refused.++Provider documentation remains the primary authority. OpenRouter is a cross-check and may lag a new model. When the development CLI is unavailable, fetch the public catalog directly and filter its `data` list:++```bash+curl -fsS 'https://openrouter.ai/api/v1/models?output_modalities=all'+```++Adding a hosted model remains a change to the hosted plane's own catalog; this tool never publishes models or modifies backend files automatically.diff --git a/mkdocs.yml b/mkdocs.ymlindex a2f76051c..5d1a732d0 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -756,6 +756,7 @@ plugins:- contributing.md: "Contributing"- contribute/configuration-defaults-and-overrides.md: "Configuration Internals"- contribute/keyword-only-arguments.md: "Keyword-Only Arguments"+ - contribute/openrouter-model-research.md: "OpenRouter Model Research"- contribute/hub-layering.md: "Hub Layering"- contribute/rich-imports.md: "Rich Imports"- contribute/actions-allowlist.md: "Actions Allowlist"@@ -1006,6 +1007,7 @@ nav:- Project:- Contributing: contributing.md- Configuration Internals: contribute/configuration-defaults-and-overrides.md+ - OpenRouter Model Research: contribute/openrouter-model-research.md- Keyword-Only Arguments: contribute/keyword-only-arguments.md- Hub Layering: contribute/hub-layering.md- Rich Imports: contribute/rich-imports.mddiff --git a/pipelex/cli/dev_cli/_dev_cli.py b/pipelex/cli/dev_cli/_dev_cli.pyindex e84c9ce5e..b306efd2f 100644--- a/pipelex/cli/dev_cli/_dev_cli.py+++ b/pipelex/cli/dev_cli/_dev_cli.py@@ -32,6 +32,7 @@ from pipelex.cli.dev_cli.commands.generate_error_identity_cmd import generate_erfrom pipelex.cli.dev_cli.commands.generate_mthds_schema_cmd import generate_mthds_schema_cmdfrom pipelex.cli.dev_cli.commands.generate_projection_corpus_cmd import generate_projection_corpus_cmdfrom pipelex.cli.dev_cli.commands.kit_cmd import kit_app+from pipelex.cli.dev_cli.commands.openrouter.openrouter_cmd import openrouter_appfrom pipelex.cli.dev_cli.commands.preprocess_test_models_cmd import preprocess_test_models_cmdfrom pipelex.cli.dev_cli.commands.refresh_graph_ui_sri_cmd import refresh_graph_ui_sri_cmdfrom pipelex.cli.dev_cli.commands.store_test_durations_cmd import store_test_durations_cmd@@ -70,6 +71,7 @@ class PipelexDevCLI(TyperGroup):"generate-mthds-schema","generate-projection-corpus","kit",+ "openrouter","preprocess-test-models","refresh-graph-ui-sri","store-test-durations",@@ -106,12 +108,13 @@ app = typer.Typer(app.add_typer(drift_app, name="drift", help="Drift contracts: deterministic review obligations between code, docs, and tests")app.add_typer(kit_app, name="kit", help="Manage agent rules for the Pipelex repository")+app.add_typer(openrouter_app, name="openrouter")@app.callback(invoke_without_command=True)def app_callback(_ctx: typer.Context) -> None:# Skip banner if --quiet or -q flag is present- if "--quiet" in sys.argv or "-q" in sys.argv:+ if "--quiet" in sys.argv or "-q" in sys.argv or _ctx.invoked_subcommand == "openrouter":returnconsole = get_console()diff --git a/pipelex/cli/dev_cli/commands/openrouter/__init__.py b/pipelex/cli/dev_cli/commands/openrouter/__init__.pynew file mode 100644index 000000000..f63d719f7--- /dev/null+++ b/pipelex/cli/dev_cli/commands/openrouter/__init__.py@@ -0,0 +1 @@+"""OpenRouter catalog research for Pipelex contributors."""diff --git a/pipelex/cli/dev_cli/commands/openrouter/catalog.py b/pipelex/cli/dev_cli/commands/openrouter/catalog.pynew file mode 100644index 000000000..c7958e38b--- /dev/null+++ b/pipelex/cli/dev_cli/commands/openrouter/catalog.py@@ -0,0 +1,67 @@+"""Read the public catalog without depending on hosted-plane configuration."""++import os+from typing import Any+from urllib.parse import urljoin, urlsplit++import httpx+from pydantic import BaseModel, ConfigDict, Field++CATALOG_URL = "https://openrouter.ai/api/v1/models"+++class OpenRouterArchitecture(BaseModel):+ model_config = ConfigDict(extra="allow", strict=True)++ input_modalities: list[str] = Field(default_factory=list)+ output_modalities: list[str] = Field(default_factory=list)+++class OpenRouterModel(BaseModel):+ model_config = ConfigDict(extra="allow", strict=True)++ id: str = Field(min_length=1)+ name: str = ""+ created: int | None = None+ context_length: int | None = None+ architecture: OpenRouterArchitecture | None = None+ pricing: dict[str, Any] | None = None+ top_provider: dict[str, Any] | None = None+ supported_parameters: list[str] = Field(default_factory=list)+++class CatalogPage(BaseModel):+ model_config = ConfigDict(strict=True)++ data: list[OpenRouterModel]+ links: dict[str, str | None] | None = None+++def fetch_models(*, category: str | None = None) -> list[OpenRouterModel]:+ """Fetch all modalities, retaining the first occurrence of each model ID."""+ headers = {"Accept": "application/json"}+ if api_key := os.getenv("OPENROUTER_API_KEY"):+ headers["Authorization"] = f"Bearer {api_key}"+ initial_params = {"output_modalities": "all"}+ if category:+ initial_params["category"] = category+ params: dict[str, str] | None = initial_params+ models: dict[str, OpenRouterModel] = {}+ seen_pages: set[str] = set()+ next_url: str | None = CATALOG_URL+ with httpx.Client(timeout=30.0, headers=headers) as client:+ while next_url:+ parsed = urlsplit(next_url)+ if (parsed.scheme, parsed.netloc, parsed.path) != ("https", "openrouter.ai", "/api/v1/models") or next_url in seen_pages:+ msg = "OpenRouter returned an invalid catalog pagination link"+ raise ValueError(msg)+ seen_pages.add(next_url)+ response = client.get(next_url, params=params)+ response.raise_for_status()+ page = CatalogPage.model_validate(response.json())+ for model in page.data:+ models.setdefault(model.id, model)+ next_link = (page.links or {}).get("next")+ next_url = urljoin(CATALOG_URL, next_link) if next_link else None+ params = None+ return list(models.values())diff --git a/pipelex/cli/dev_cli/commands/openrouter/export.py b/pipelex/cli/dev_cli/commands/openrouter/export.pynew file mode 100644index 000000000..d339b06a7--- /dev/null+++ b/pipelex/cli/dev_cli/commands/openrouter/export.py@@ -0,0 +1,72 @@+"""Render disabled draft backend entries using the runtime's current vocabulary."""++from decimal import Decimal, InvalidOperation++import tomlkit++from pipelex.cli.dev_cli.commands.openrouter.catalog import OpenRouterModel+from pipelex.cogt.model_backends.model_spec_document import describe_model_spec_document_rejection++INPUT_MODALITIES = {"text": "text", "image": "images", "file": "pdf", "audio": "audio", "video": "video"}+++def price_per_million(*, price: object) -> float | None:+ if not isinstance(price, str):+ return None+ try:+ value = Decimal(price) * 1_000_000+ if not value.is_finite() or value < 0:+ return None+ converted = float(value)+ if converted == float("inf"):+ return None+ return converted+ except (InvalidOperation, ValueError, OverflowError):+ return None+++def models_to_toml(*, models: list[OpenRouterModel]) -> str:+ document = tomlkit.document()+ document.add(tomlkit.comment("Draft OpenRouter backend. Review routing, capabilities and prices before enabling each entry."))+ document.add(tomlkit.comment("Token prices are USD per million tokens. Missing costs remain unknown; image/request fees need review."))+ for model in models:+ architecture = model.architecture+ if architecture is None:+ msg = f"Model {model.id!r} has no architecture; cannot export a backend entry"+ raise ValueError(msg)+ outputs = architecture.output_modalities+ if not outputs or any(output not in {"text", "image"} for output in outputs):+ msg = f"Model {model.id!r} has unsupported output modalities; export text and image models separately with --filter"+ raise ValueError(msg)+ entry = tomlkit.table()+ entry.add("enabled", False)+ if "image" in outputs:+ entry.add("model_type", "img_gen")+ entry.add("sdk", "openrouter_img_gen")+ else:+ entry.add("model_type", "llm")+ entry.add("sdk", "openai")+ inputs = [INPUT_MODALITIES[modality] for modality in architecture.input_modalities if modality in INPUT_MODALITIES]+ entry.add("inputs", inputs)+ exported_outputs = list(outputs)+ if "image" not in outputs and "structured_outputs" in model.supported_parameters:+ exported_outputs.append("structured")+ entry.add("structure_method", "instructor/openrouter_structured_outputs")+ entry.add("outputs", exported_outputs)+ # A context window is not an output limit. Omit an unknown limit.+ max_completion_tokens = (model.top_provider or {}).get("max_completion_tokens")+ if isinstance(max_completion_tokens, int) and not isinstance(max_completion_tokens, bool) and max_completion_tokens > 0:+ entry.add("max_tokens", max_completion_tokens)+ pricing = model.pricing or {}+ costs = tomlkit.inline_table()+ for source, destination in (("prompt", "input"), ("completion", "output")):+ price = price_per_million(price=pricing.get(source))+ if price is not None:+ costs.add(destination, price)+ if costs:+ entry.add("costs", costs)+ document.add(model.id, entry)+ rejection = describe_model_spec_document_rejection(document=document.unwrap())+ if rejection:+ raise ValueError(rejection)+ return document.as_string()diff --git a/pipelex/cli/dev_cli/commands/openrouter/openrouter_cmd.py b/pipelex/cli/dev_cli/commands/openrouter/openrouter_cmd.pynew file mode 100644index 000000000..9e3751658--- /dev/null+++ b/pipelex/cli/dev_cli/commands/openrouter/openrouter_cmd.py@@ -0,0 +1,132 @@+"""Look up public model facts and export draft backend TOML."""++import json+from enum import StrEnum+from pathlib import Path+from typing import Annotated++import httpx+import typer+from pydantic import ValidationError++from pipelex.cli.dev_cli.commands.openrouter.catalog import OpenRouterModel, fetch_models+from pipelex.cli.dev_cli.commands.openrouter.export import models_to_toml, price_per_million++openrouter_app = typer.Typer(no_args_is_help=True, help="Research OpenRouter models and export draft backend entries.")+++class ModelSortKey(StrEnum):+ NAME = "name"+ CONTEXT_LENGTH = "context_length"+ CREATED = "created"+++def sort_models(*, models: list[OpenRouterModel], sort_key: ModelSortKey) -> list[OpenRouterModel]:+ match sort_key:+ case ModelSortKey.NAME:+ return sorted(models, key=lambda model: (model.name or model.id).casefold())+ case ModelSortKey.CONTEXT_LENGTH:+ return sorted(models, key=lambda model: model.context_length or 0, reverse=True)+ case ModelSortKey.CREATED:+ return sorted(models, key=lambda model: model.created or 0, reverse=True)+++def markdown_cell(*, value: str) -> str:+ return value.replace("\\", "\\\\").replace("|", "\\|").replace("\n", " ").replace("\r", " ")+++def render_models(*, models: list[OpenRouterModel]) -> str:+ lines = [+ "# OpenRouter models",+ "",+ "| ID | Name | Context tokens | Input USD/M tokens | Output USD/M tokens |",+ "| --- | --- | ---: | ---: | ---: |",+ ]+ for model in models:+ pricing = model.pricing or {}+ input_price = price_per_million(price=pricing.get("prompt"))+ output_price = price_per_million(price=pricing.get("completion"))+ cells = [+ markdown_cell(value=model.id),+ markdown_cell(value=model.name),+ str(model.context_length) if model.context_length is not None else "Unknown",+ f"{input_price:g}" if input_price is not None else "Unknown",+ f"{output_price:g}" if output_price is not None else "Unknown",+ ]+ lines.append("| " + " | ".join(cells) + " |")+ if not models:+ lines.extend(["", "No models matched."])+ return "\n".join(lines)+++@openrouter_app.command("list-models")+def list_models_command(+ category: Annotated[str | None, typer.Option("--category", "-c", help="OpenRouter catalog category filter.")] = None,+ name_filter: Annotated[str | None, typer.Option("--filter", "-f", help="Match a substring of the name or ID, ignoring case.")] = None,+ sort: Annotated[+ ModelSortKey, typer.Option("--sort", "-s", help="Sort by name, context length (context_length), or creation date.")+ ] = ModelSortKey.NAME,+ output_json: Annotated[bool, typer.Option("--json", "-j", help="Emit a JSON envelope instead of Markdown.")] = False,+ save: Annotated[bool, typer.Option("--save", help="Write disabled draft TOML into generated/; refuse to overwrite an existing file.")] = False,+) -> None:+ try:+ models = fetch_models(category=category)+ if name_filter:+ needle = name_filter.casefold()+ models = [model for model in models if needle in model.id.casefold() or needle in model.name.casefold()]+ models = sort_models(models=models, sort_key=sort)+ saved_to: Path | None = None+ skipped_model_ids: list[str] = []+ if save:+ if not models:+ msg = "No models matched; no export was written"+ raise ValueError(msg)+ suffix_parts = [part for part in (category, name_filter) if part]+ suffix = "".join("_" + "".join(char if char.isalnum() or char in {"-", "_"} else "_" for char in part) for part in suffix_parts)+ saved_to = Path("generated") / f"openrouter_models{suffix}.toml"+ export_models: list[OpenRouterModel] = []+ for model in models:+ outputs = model.architecture.output_modalities if model.architecture else []+ if outputs and all(output in {"text", "image"} for output in outputs):+ export_models.append(model)+ else:+ skipped_model_ids.append(model.id)+ if not export_models:+ msg = "No matched models have exportable text or image output; no export was written"+ raise ValueError(msg)+ content = models_to_toml(models=export_models)+ saved_to.parent.mkdir(parents=True, exist_ok=True)+ with saved_to.open("x", encoding="utf-8") as output_file:+ output_file.write(content)+ except (httpx.HTTPError, ValueError, OSError) as exc:+ if isinstance(exc, ValidationError):+ message = "OpenRouter returned malformed catalog metadata"+ elif isinstance(exc, httpx.HTTPStatusError):+ message = f"OpenRouter catalog request failed with HTTP {exc.response.status_code}"+ elif isinstance(exc, httpx.HTTPError):+ message = "Could not reach the OpenRouter catalog"+ else:+ message = str(exc)+ if output_json:+ typer.echo(json.dumps({"ok": False, "error": message}), err=True)+ else:+ typer.echo(f"# OpenRouter lookup failed\n\n{message}", err=True)+ raise typer.Exit(1) from exc+ if output_json:+ typer.echo(+ json.dumps(+ {+ "ok": True,+ "models": [model.model_dump(exclude_unset=True) for model in models],+ "saved_to": str(saved_to) if saved_to else None,+ "skipped_model_ids": skipped_model_ids,+ },+ indent=2,+ )+ )+ else:+ typer.echo(render_models(models=models))+ if saved_to:+ typer.echo(f"\nDraft saved to `{saved_to}`. Review each entry before enabling it.")+ if skipped_model_ids:+ typer.echo("\nSkipped models with unsupported output modalities: " + ", ".join(skipped_model_ids))diff --git a/tests/unit/pipelex/cli/dev_cli/openrouter/__init__.py b/tests/unit/pipelex/cli/dev_cli/openrouter/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/tests/unit/pipelex/cli/dev_cli/openrouter/conftest.py b/tests/unit/pipelex/cli/dev_cli/openrouter/conftest.pynew file mode 100644index 000000000..e8fe0761f--- /dev/null+++ b/tests/unit/pipelex/cli/dev_cli/openrouter/conftest.py@@ -0,0 +1,39 @@+from typing import Any++import pytest+from pytest_mock import MockerFixture++from pipelex.cli.dev_cli.commands.openrouter.catalog import OpenRouterModel+++def catalog_models() -> list[dict[str, Any]]:+ return [+ {+ "id": "acme/text.v1:free",+ "name": "Text | Model",+ "created": 100,+ "context_length": 32000,+ "architecture": {"input_modalities": ["text", "image", "file"], "output_modalities": ["text"]},+ "pricing": {"prompt": "0.000003", "completion": "0.000015"},+ "top_provider": {"max_completion_tokens": 4096},+ "supported_parameters": ["tools"],+ },+ {+ "id": "acme/image",+ "name": "Image Model",+ "created": 200,+ "context_length": 8000,+ "architecture": {"input_modalities": ["text", "image"], "output_modalities": ["image"]},+ "pricing": {"prompt": None, "completion": "-1"},+ "supported_parameters": [],+ },+ ]+++def mocked_catalog(mocker: MockerFixture, catalog_models: list[dict[str, Any]]) -> None:+ mocker.patch(+ "pipelex.cli.dev_cli.commands.openrouter.openrouter_cmd.fetch_models",+ return_value=[OpenRouterModel.model_validate(model) for model in catalog_models],+ )diff --git a/tests/unit/pipelex/cli/dev_cli/openrouter/test_catalog.py b/tests/unit/pipelex/cli/dev_cli/openrouter/test_catalog.pynew file mode 100644index 000000000..7aab32d95--- /dev/null+++ b/tests/unit/pipelex/cli/dev_cli/openrouter/test_catalog.py@@ -0,0 +1,44 @@+"""Catalog transport, command output and validated backend exports."""++from typing import Any++import httpx+import pytest+from pytest_mock import MockerFixture++from pipelex.cli.dev_cli.commands.openrouter.catalog import fetch_models+++class TestOpenRouterCatalog:+ def test_all_modalities_auth_category_pagination(self, mocker: MockerFixture, catalog_models: list[dict[str, Any]]) -> None:+ mocker.patch.dict("os.environ", {"OPENROUTER_API_KEY": "test-key"})+ requests: list[httpx.Request] = []++ def respond(request: httpx.Request) -> httpx.Response:+ requests.append(request)+ if len(requests) == 1:+ return httpx.Response(200, json={"data": catalog_models[:1], "links": {"next": "/api/v1/models?offset=1"}})+ return httpx.Response(200, json={"data": catalog_models, "links": {"next": None}})++ client = httpx.Client(transport=httpx.MockTransport(respond))+ constructor = mocker.patch("pipelex.cli.dev_cli.commands.openrouter.catalog.httpx.Client", return_value=client)+ models = fetch_models(category="programming")+ assert [model.id for model in models] == [model["id"] for model in catalog_models]+ assert requests[0].url.params["output_modalities"] == "all"+ assert requests[0].url.params["category"] == "programming"+ assert requests[1].url.params["offset"] == "1"+ assert constructor.call_args.kwargs["headers"]["Authorization"] == "Bearer test-key"++ def test_anonymous_lookup(self, mocker: MockerFixture) -> None:+ mocker.patch.dict("os.environ", {}, clear=True)+ client = httpx.Client(transport=httpx.MockTransport(lambda _request: httpx.Response(200, json={"data": []})))+ constructor = mocker.patch("pipelex.cli.dev_cli.commands.openrouter.catalog.httpx.Client", return_value=client)+ assert fetch_models() == []+ assert "Authorization" not in constructor.call_args.kwargs["headers"]++ @pytest.mark.parametrize("next_link", ["https://attacker.example/models", "https://openrouter.ai/api/v1/models"])+ def test_rejects_unsafe_or_repeated_pages(self, mocker: MockerFixture, next_link: str) -> None:+ client = httpx.Client(transport=httpx.MockTransport(lambda _request: httpx.Response(200, json={"data": [], "links": {"next": next_link}})))+ mocker.patch("pipelex.cli.dev_cli.commands.openrouter.catalog.httpx.Client", return_value=client)+ with pytest.raises(ValueError, match="pagination"):+ fetch_models()diff --git a/tests/unit/pipelex/cli/dev_cli/openrouter/test_command.py b/tests/unit/pipelex/cli/dev_cli/openrouter/test_command.pynew file mode 100644index 000000000..78d08d37b--- /dev/null+++ b/tests/unit/pipelex/cli/dev_cli/openrouter/test_command.py@@ -0,0 +1,94 @@+"""Catalog transport, command output and validated backend exports."""++import json+from pathlib import Path+from typing import Any++import click+import httpx+import pytest+import tomli+from pytest_mock import MockerFixture+from typer.testing import CliRunner++from pipelex.cli.dev_cli._dev_cli import app+from pipelex.cli.dev_cli.commands.openrouter.catalog import OpenRouterModel+++class TestOpenRouterCommand:+ @pytest.mark.usefixtures("mocked_catalog")+ def test_json_filter(self) -> None:+ result = CliRunner().invoke(app, ["openrouter", "list-models", "-f", "TEXT.V1", "--json"])+ assert result.exit_code == 0, result.output+ payload = json.loads(result.stdout)+ assert payload["ok"] is True+ assert [model["id"] for model in payload["models"]] == ["acme/text.v1:free"]+ assert not result.stderr++ @pytest.mark.usefixtures("mocked_catalog")+ @pytest.mark.parametrize(("sort_key", "first_id"), [("name", "acme/image"), ("context_length", "acme/text.v1:free"), ("created", "acme/image")])+ def test_sort_json(self, sort_key: str, first_id: str) -> None:+ result = CliRunner().invoke(app, ["openrouter", "list-models", "--sort", sort_key, "--json"])+ assert result.exit_code == 0, result.output+ assert json.loads(result.stdout)["models"][0]["id"] == first_id++ @pytest.mark.usefixtures("mocked_catalog")+ def test_markdown(self) -> None:+ result = CliRunner().invoke(app, ["openrouter", "list-models"])+ assert result.exit_code == 0, result.output+ assert "| Context tokens |" in result.stdout+ assert r"Text \| Model" in result.stdout+ assert "Unknown" in result.stdout+ assert "Internal Development Tools" not in result.stdout++ @pytest.mark.usefixtures("mocked_catalog")+ def test_export_refuses_overwrite(self) -> None:+ runner = CliRunner()+ with runner.isolated_filesystem():+ args = ["openrouter", "list-models", "--save", "--json", "-f", "text.v1"]+ first = runner.invoke(app, args)+ assert first.exit_code == 0, first.output+ path = Path(json.loads(first.stdout)["saved_to"])+ original = path.read_text(encoding="utf-8")+ second = runner.invoke(app, args)+ assert second.exit_code == 1+ assert not second.stdout+ assert json.loads(second.stderr)["ok"] is False+ assert path.read_text(encoding="utf-8") == original++ def test_export_reports_skipped_modalities(self, mocker: MockerFixture, catalog_models: list[dict[str, Any]]) -> None:+ models = [OpenRouterModel.model_validate(model) for model in catalog_models]+ models.append(OpenRouterModel.model_validate({"id": "acme/audio", "architecture": {"output_modalities": ["speech"]}}))+ mocker.patch("pipelex.cli.dev_cli.commands.openrouter.openrouter_cmd.fetch_models", return_value=models)+ runner = CliRunner()+ with runner.isolated_filesystem():+ result = runner.invoke(app, ["openrouter", "list-models", "--save", "--json"])+ assert result.exit_code == 0, result.output+ payload = json.loads(result.stdout)+ assert payload["skipped_model_ids"] == ["acme/audio"]+ assert len(payload["models"]) == len(models)+ assert "acme/audio" not in tomli.loads(Path(payload["saved_to"]).read_text(encoding="utf-8"))++ @pytest.mark.parametrize("payload", [{"wrong": []}, {"data": [{"id": 123}]}, {"data": "invalid"}])+ def test_malformed_response(self, mocker: MockerFixture, payload: dict[str, Any]) -> None:+ client = httpx.Client(transport=httpx.MockTransport(lambda _request: httpx.Response(200, json=payload)))+ mocker.patch("pipelex.cli.dev_cli.commands.openrouter.catalog.httpx.Client", return_value=client)+ result = CliRunner().invoke(app, ["openrouter", "list-models", "--json"])+ assert result.exit_code == 1+ assert not result.stdout+ assert json.loads(result.stderr) == {"ok": False, "error": "OpenRouter returned malformed catalog metadata"}++ def test_http_failure(self, mocker: MockerFixture) -> None:+ client = httpx.Client(transport=httpx.MockTransport(lambda _request: httpx.Response(503, text="private upstream body")))+ mocker.patch("pipelex.cli.dev_cli.commands.openrouter.catalog.httpx.Client", return_value=client)+ result = CliRunner().invoke(app, ["openrouter", "list-models"])+ assert result.exit_code == 1+ assert "HTTP 503" in result.stderr+ assert "private upstream body" not in result.output++ @pytest.mark.parametrize("color", [False, True])+ def test_help(self, color: bool) -> None:+ assert "openrouter" in CliRunner().invoke(app, ["--help"]).stdout+ result = CliRunner().invoke(app, ["openrouter", "list-models", "--help"], color=color)+ assert result.exit_code == 0+ assert "--save" in click.unstyle(result.stdout)diff --git a/tests/unit/pipelex/cli/dev_cli/openrouter/test_export.py b/tests/unit/pipelex/cli/dev_cli/openrouter/test_export.pynew file mode 100644index 000000000..5b0540898--- /dev/null+++ b/tests/unit/pipelex/cli/dev_cli/openrouter/test_export.py@@ -0,0 +1,43 @@+"""Catalog transport, command output and validated backend exports."""++from typing import Any++import pytest+import tomli++from pipelex.cli.dev_cli.commands.openrouter.catalog import OpenRouterModel+from pipelex.cli.dev_cli.commands.openrouter.export import models_to_toml+from pipelex.cogt.model_backends.model_spec_document import describe_model_spec_document_rejection+++class TestOpenRouterExport:+ def test_validated_drafts(self, catalog_models: list[dict[str, Any]]) -> None:+ models = [OpenRouterModel.model_validate(model) for model in catalog_models]+ document = tomli.loads(models_to_toml(models=models))+ assert describe_model_spec_document_rejection(document=document) is None+ text = document["acme/text.v1:free"]+ assert text["max_tokens"] == 4096+ assert text["enabled"] is False+ assert text["inputs"] == ["text", "images", "pdf"]+ assert text["outputs"] == ["text"]+ assert "structure_method" not in text+ assert text["costs"] == {"input": 3.0, "output": 15.0}+ image = document["acme/image"]+ assert image["sdk"] == "openrouter_img_gen"+ assert "costs" not in image+ assert "max_tokens" not in image++ def test_explicit_structured_and_quoted_key(self, catalog_models: list[dict[str, Any]]) -> None:+ model = OpenRouterModel.model_validate(catalog_models[0])+ model.id = 'vendor/quoted"name'+ model.supported_parameters = ["structured_outputs"]+ document = tomli.loads(models_to_toml(models=[model]))+ assert document[model.id]["outputs"] == ["text", "structured"]+ assert describe_model_spec_document_rejection(document=document) is None++ @pytest.mark.parametrize("price", [None, "-1", "NaN", "Infinity", "not a price"])+ def test_unknown_prices(self, catalog_models: list[dict[str, Any]], price: str | None) -> None:+ model = OpenRouterModel.model_validate(catalog_models[0])+ model.pricing = {"prompt": price}+ document = tomli.loads(models_to_toml(models=[model]))+ assert "costs" not in document[model.id]