Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

refactor/Drop-build-routes · L-260829-848001 (#1328)

Louis Choquela3034bc2dcd9
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d64fe1983..513f4f3c3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -28,6 +28,10 @@
- **The logging documentation no longer promises that a server refuses the `console` sink**: the log-sink, logging, logging-configuration and Rich-imports pages said a server installed without the `cli` extra stops at boot when it selects the `console` sink or the `rich` pretty-print mode. The core dependencies `typer` and `instructor` install Rich anyway, so such a server boots on the Rich surface; the pages now say a server selects `json` and a Rich-free mode in its own configuration, since nothing at boot makes that choice for it.
- **`pipelex doctor` checks the backends against the configured secrets provider**: the backend-file probe and the Models row resolved credentials from environment variables whatever `[runtime.secrets] method` selected, so a deployment on another secrets method was diagnosed against secrets it does not use. Both now use the provider the configuration selects, and say they could not run when it does not build; a provider that raises on a lookup, an unreachable vault or a missing SDK, is reported in the Models row rather than stopping the doctor.
+### Removed
+
+- **The API server's `/v1/build/*` routes (Breaking)**: `pipelex-api` no longer serves `POST /v1/build/inputs`, `/v1/build/output`, `/v1/build/runner`, `/v1/build/concept` or `/v1/build/pipe-spec`, which now answer `404`, and the `MAX_AGENT_SPEC_KIB` setting that bounded the last two is gone. A client reads a pipe's I/O from `POST /v1/pipe-io` and projects an inputs template from its `input_form` with the `mthds` packages' projection (`renderInputsTemplate`, which `@pipelex/sdk` re-exports, or `mthds.protocol.inputs_template` in Python), as `mthds-agent inputs` does; the output representation, the runner script and the concept and pipe spec conversions have no HTTP replacement.
+
### Security
- **Every template render spends from a budget (Breaking)**: a render is charged for what it reads, allocates and prints, and an operation that would overdraw its budget, such as `'x' * 10 ** 9`, a huge format width, `lipsum(n=10 ** 9)`, nested empty loops or a runaway recursive macro, is refused before it allocates and raises `Jinja2TemplateBudgetError`. Converting Markdown to HTML, with the `markdown` filter or by printing a Markdown value in an HTML template, is charged for every character and table cell of its source, so one render converts at most about 65,000 characters of Markdown. Template arithmetic stops at 64-bit integers, and a template can no longer call `str.encode` or `int.to_bytes`. A filter or test registered on a template environment needs a cost in the render budget's tables, or the first compile raises a `TypeError` naming it, and Pipelex now requires `jinja2` below 3.2.
diff --git a/api/.claude/commands/update-postman.md b/api/.claude/commands/update-postman.md
index 94b9c9037..520bfe32e 100644
--- a/api/.claude/commands/update-postman.md
+++ b/api/.claude/commands/update-postman.md
@@ -91,9 +91,9 @@ Compare the Postman baseline (step 1) against the code (step 3):
Produce a clear **change plan** and show it to the user before proceeding:
```
Changes to apply:
- + ADD: POST /v1/new-endpoint (folder: Build)
- ~ UPDATE: POST /v1/build/pipe-spec — request body changed (added "model" field)
- - REMOVE: POST /v1/build/pipe — endpoint deleted
+ + ADD: POST /v1/new-endpoint (folder: Validate)
+ ~ UPDATE: POST /v1/validate — request body changed (added "render" field)
+ - REMOVE: POST /v1/old-endpoint — endpoint deleted
= UNCHANGED: 28 requests
```
@@ -146,11 +146,6 @@ For **unchanged endpoints**, do NOT touch them.
|---|---|
| Pipeline Execute/Start | 1 per pipe type, 1 pipe_code only, 1 error |
| Validate | 1 valid, 1 missing main_pipe error |
-| Build Inputs | 1 happy path, 1 pipe not found error |
-| Build Output | 1 per format (`schema`, `json`, `python`) |
-| Build Runner | 1 happy path, 1 error |
-| Build Concept | structured fields, refines, concept_ref, 1 error |
-| Build Pipe Spec | 1 per pipe type, 1 invalid type error |
| Models | no filter, filter by `llm`, filter by `extract`, multi-type |
| Presigned Post URLs | single file, multiple files |
| Version | version (protocol handshake) |
@@ -169,13 +164,11 @@ with open('/tmp/postman_current.json') as f:
col = data['collection']
# Navigate to the right folder
-# Example: find Agent > Build Pipe Spec
+# Example: find the Validate folder
for folder in col['item']:
- if folder['name'] == 'Agent':
- for sub in folder['item']:
- if sub['name'] == 'Build Pipe Spec':
- target = sub
- break
+ if folder['name'] == 'Validate':
+ target = folder
+ break
# Build the new request item
new_item = {
@@ -188,16 +181,16 @@ new_item = {
"raw": json.dumps({...}, indent="\t")
},
"url": {
- "raw": "{{base_url}}/v1/build/pipe-spec",
+ "raw": "{{base_url}}/v1/validate",
"host": ["{{base_url}}"],
- "path": ["v1", "build", "pipe-spec"]
+ "path": ["v1", "validate"]
},
"description": "..."
},
"response": [
{
"name": "200 OK",
- "originalRequest": {"method": "POST", "url": {"raw": "{{base_url}}/v1/build/pipe-spec", "host": ["{{base_url}}"], "path": ["v1", "build", "pipe-spec"]}},
+ "originalRequest": {"method": "POST", "url": {"raw": "{{base_url}}/v1/validate", "host": ["{{base_url}}"], "path": ["v1", "validate"]}},
"status": "OK",
"code": 200,
"body": json.dumps({...})
@@ -237,8 +230,7 @@ Key patterns:
- `Version` → version (protocol handshake)
- `Pipeline` → `Execute (sync)`, `Start (async)`
- `Validate` → with valid/invalid sub-examples
-- `Build` → `Build Inputs`, `Build Output`, `Build Runner`
-- `Agent` → `Build Concept`, `Build Pipe Spec`, `Models`
+- `Agent` → `Models`
- `Uploader` → `Presigned Post URLs`
### 9. Push to Postman
diff --git a/api/.claude/skills/postman-bundle/SKILL.md b/api/.claude/skills/postman-bundle/SKILL.md
index 174fdcc6a..edd5c0c2e 100644
--- a/api/.claude/skills/postman-bundle/SKILL.md
+++ b/api/.claude/skills/postman-bundle/SKILL.md
@@ -5,13 +5,12 @@ description: >-
as a ready-to-run query into the live "Pipelex FastAPI" Postman collection, emit a curl command,
execute it directly, or just preview the body. Covers the whole bundle surface: run (/v1/execute,
/v1/start), dry-run validate (/v1/validate — no inference, no cost), crate resolution (/v1/resolve),
- typed codegen (/v1/codegen — ts-zod, python-pydantic, python-structures), and the per-pipe build
- projections (/v1/build/inputs, /v1/build/output, /v1/build/runner). Use this whenever the user points
+ and typed codegen (/v1/codegen — ts-zod, python-pydantic, python-structures). Use this whenever the user points
at a bundle directory or a .mthds file (often with an inputs.json) and wants to run, validate,
resolve, codegen, or otherwise test it via the API — e.g. "make a Postman query for this bundle",
"run the fashion_moodboard bundle against the API", "validate this bundle via the API", "resolve this
bundle to a crate", "codegen this bundle", "generate the TypeScript types for this method over the
- API", "test /v1/resolve on this bundle", "build the inputs template via the API", "give me the curl
+ API", "test /v1/resolve on this bundle", "give me the curl
for this bundle", or just "postman" said alongside a bundle path. It resolves the bundle exactly like
`pipelex run bundle <path>` (auto-detects bundle.mthds / the single .mthds, reads main_pipe, loads the
sibling inputs.json). Trigger it even when the user does not name the endpoint or the word "skill".
@@ -32,9 +31,6 @@ This is a focused, one-bundle-at-a-time tool. It is **not** the full collection
| `validate` | `POST /v1/validate` | `mthds_contents` | no | no | no (free dry-run) |
| `resolve` | `POST /v1/resolve` | `files[]` crate | no | no | no (free) |
| `codegen` | `POST /v1/codegen` | `files[]` crate + `kind`/`target` | no | no | no (free) |
-| `build-inputs` | `POST /v1/build/inputs` | `files[]` + `pipe_ref` + `format`/`explicit` | optional | no | no (free) |
-| `build-output` | `POST /v1/build/output` | `files[]` + `pipe_ref` + `format` | optional | no | no (free) |
-| `build-runner` | `POST /v1/build/runner` | `files[]` + `pipe_ref` + `allow_signatures` | optional | no | no (free) |
Every non-run endpoint answers with the **`/validate` verdict discipline**: a produced verdict is always a `200` discriminated on `is_valid` — the artifact on the valid arm, structured `validation_errors[]` on the invalid arm. Non-2xx is reserved for no-verdict conditions (request-shape `422`, auth `401`/`403`, server `5xx`), rendered as RFC 7807 `problem+json`.
@@ -47,7 +43,6 @@ Bundles/
<bundle>/ (named by the bundle's `domain`, or the filename)
Execute (sync) Start (async) Validate (dry-run)
Resolve (crate) Codegen (types → <target>)
- Build Inputs Build Output (<format>) Build Runner
```
Which requests appear depends on `--endpoint`: the default `both` is `execute` + `start`; every other endpoint is pushed alone. Pushes **merge by request name** — pushing `--endpoint codegen` doesn't wipe an Execute/Start pair pushed earlier, and two codegen targets (distinct names) coexist. Requests use the collection variable `{{base_url}}` and inherit the collection's `{{auth_token}}` bearer auth (and the **Start** request's `callback_urls` is the `{{callback_url}}` variable), so the same query works against a local or hosted server by switching the Postman environment.
@@ -69,7 +64,7 @@ Which requests appear depends on `--endpoint`: the default `both` is `execute` +
(`pipe_code` and `inputs` ride execute/start only; `allow_signatures` rides validate.)
-**Crate + build routes** (`resolve`, `codegen`, `build-*`) pair each content with its source in one `files[]` entry — the envelope the codegen spec pins. No `inputs`; the closure is the whole request. The build routes add a `pipe_ref` (the **qualified** `domain.pipe_code`; optional on the wire, defaulting to the closure's `main_pipe` — this skill always sends it explicitly):
+**Crate routes** (`resolve`, `codegen`) pair each content with its source in one `files[]` entry — the envelope the codegen spec pins. No `inputs`; the closure is the whole request:
```json
{
@@ -79,21 +74,21 @@ Which requests appear depends on `--endpoint`: the default `both` is `execute` +
}
```
-(`kind`/`target` are codegen-only; `pipe_ref` is build-only.) The **start** body additionally carries `callback_urls` (see [Callback URLs](#callback-urls-the-async-start-endpoint)). The **validate** body optionally carries `render` (see below).
+(`kind`/`target` are codegen-only.) The **start** body additionally carries `callback_urls` (see [Callback URLs](#callback-urls-the-async-start-endpoint)). The **validate** body optionally carries `render` (see below).
## Validate = the API's dry-run
`POST /v1/validate` is the API's dry-run: it parses, loads, and dry-runs every pipe with mock inputs and **zero inference** (no LLM calls, no cost), then returns the validated bundle blueprint, a `graph_spec`, and per-pipe input/output JSON-Schema `pipe_io_contracts`. It confirms the whole pipeline wires up without ever running it — the safe, free counterpart to `execute`/`start`. Reach for it to "just check" or "dry-run" a bundle.
- **`main_pipe` is optional for `/validate`.** A bundle with no `main_pipe` still validates (200, `is_valid: true`) — you just get `graph_spec: null`. `--pipe` is ignored.
-- **`allow_signatures`** (`--allow-signatures`) tolerates unimplemented pipe signatures instead of rejecting the bundle (each dry-runs trivially by minting a mock). Useful for in-progress bundles. It parameterizes the **dry-run sweep**, so it rides `validate` and `build-runner` only — the static `build-inputs`/`build-output` projections dropped the sweep and do not accept it. Default strict.
+- **`allow_signatures`** (`--allow-signatures`) tolerates unimplemented pipe signatures instead of rejecting the bundle (each dry-runs trivially by minting a mock). Useful for in-progress bundles. It parameterizes the **dry-run sweep**, so it rides `validate` only — the crate routes run no sweep and do not accept it. Default strict.
- **`render`** (`--render markdown`) opts into a server-rendered Markdown view of the verdict, riding the 200 on both arms as `rendered_markdown`. This is what a **skill**-driven validate sends; a **hook** omits it and reads the structured `is_valid`. The structured fields stay the contract; the Markdown is the view.
## Resolve & Codegen (the crate routes)
**`POST /v1/resolve`** resolves the library closure into its **normalized crate** — fully qualified refs, flattened refinement, materialized natives, fingerprint set. It runs no dry-run sweep (runnability is `/validate`'s vocabulary). The valid arm carries the canonical JSON crate — the same bytes `pipelex resolve --format json` prints, so a fingerprint computed from either surface agrees.
-**`POST /v1/codegen`** resolves the closure the same way, then projects the crate through two explicit axes: `kind` (only `types` is served — input templates ride `/build/inputs` instead) × `target`:
+**`POST /v1/codegen`** resolves the closure the same way, then projects the crate through two explicit axes: `kind` (only `types` is served — an inputs template is projected client-side from the input form `POST /v1/pipe-io` serves) × `target`:
- `ts-zod` — zod schemas + inferred TypeScript types
- `python-pydantic` — self-contained pydantic BaseModels
@@ -101,17 +96,7 @@ Which requests appear depends on `--endpoint`: the default `both` is `execute` +
The valid arm carries the **stamped** artifact set plus its `codegen.lock`: a client that writes them verbatim reproduces a local `pipelex codegen types` run **byte-for-byte** and passes the offline `pipelex codegen check`. There is deliberately no server-side check route (the drift check is offline by design). An unknown `kind`/`target`, or a `pipe_ref` on the concept-set-wide `types`, is a request-shape `422`.
-`--target` is **required** for codegen — there is no default; ask the user which consumer they're generating for if unclear. The request envelope also accepts a `method_ref` instead of `files[]`, but the server answers `501` until server-side method-registry resolution exists — this skill always sends `files[]`.
-
-## Build routes (per-pipe projections)
-
-`POST /v1/build/{inputs,output,runner}` project one artifact for one pipe of the closure, with no inference. They ride the same `files[]` closure selector as the crate routes plus a `pipe_ref` — resolved here from the bundle's `domain` + `main_pipe`, or from `--pipe`.
-
-`build-inputs` and `build-output` are **static**: they resolve the closure and read the pipe's *declared* IO — no dry-run sweep, so a valid verdict from them is not a promise the pipe runs (ask `validate` for that). `build-runner` keeps the sweep, because a runner script *is* that promise.
-
-- **`build-inputs`** → the inputs template for the pipe (what an `inputs.json` should look like). The valid arm carries `inputs` (a parsed object) for the default `format: json`, or `inputs_toml` (raw text) for `format: toml`; `explicit: true` swaps the light values for the ceremonial `{concept, content}` envelopes.
-- **`build-output`** → the pipe's output representation; `--output-format schema|json|python` (default `schema`). `schema`/`json` land in `output` (a parsed object); `python` lands in `output_python` (source text).
-- **`build-runner`** → a Python runner script spelling its imports with the **emitted** class names, plus the `structures` projection it imports from (stamped `structures.py` + `codegen.lock`, to write into a `structures/` directory beside the script) — matching what a local `pipelex build runner` scaffolds. A pipe recorded SKIPPED by the sweep (unresolved cross-package dependency) is a request-shape `422`.
+`--target` is **required** for codegen — there is no default; ask the user which consumer they're generating for if unclear. The request envelope also accepts a `method_ref` instead of `files[]` (an address-form reference is fetched and resolved; only the registry form answers `501`) — this skill always sends `files[]`.
## Callback URLs (the async `start` endpoint)
@@ -139,12 +124,12 @@ make bundle-codegen BUNDLE=<dir|.mthds> TARGET=ts-zod # project typed artifac
make bundle-curl BUNDLE=<dir|.mthds> # emit a ready-to-run curl
make bundle-postman BUNDLE=<dir|.mthds> # push request(s) into the Postman collection
make bundle-dry BUNDLE=<dir|.mthds> # print the request body only
-# optional: ENDPOINT=execute|start|validate|resolve|codegen|build-inputs|build-output|build-runner|both
+# optional: ENDPOINT=execute|start|validate|resolve|codegen|both
# PIPE=<code> INPUTS=<path> NAME=<folder> ALLOW_SIGNATURES=1 TARGET=<codegen target>
-# OUTPUT_FORMAT=schema|json|python CALLBACK_URL=<url> BASE_URL=<url> TOKEN=<bearer> ARGS=<extra>
+# CALLBACK_URL=<url> BASE_URL=<url> TOKEN=<bearer> ARGS=<extra>
```
-`bundle-validate` / `bundle-resolve` / `bundle-codegen` hardcode `--run` with their endpoint (they're free and safe to run live). The build routes ride the generic targets via `ENDPOINT=`, e.g. `make bundle-run ENDPOINT=build-inputs`, `make bundle-postman ENDPOINT=build-runner`, `make bundle-curl ENDPOINT=build-output OUTPUT_FORMAT=json`.
+`bundle-validate` / `bundle-resolve` / `bundle-codegen` hardcode `--run` with their endpoint (they're free and safe to run live).
Under the hood it's one script that resolves the bundle once and sends the request to one of four sinks. `<bundle-path>` is a bundle **directory** (preferred) or a single `.mthds` file. Call the script directly for a path outside the repo or finer control:
@@ -171,7 +156,6 @@ The API must be up — start it locally with `make run` first. **Heads-up: only
python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint validate
python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint resolve
python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint codegen --target python-pydantic
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint build-inputs
```
**Emit a curl command** (`--curl`) — writes the body to a temp file and prints a `curl --data @file`, so the multi-line contents never need shell escaping. **Preview only** (`--dry-run`) — print the request body, touch nothing.
@@ -182,11 +166,10 @@ Useful options (apply across modes):
|---|---|
| `--endpoint <ep>` | Pick the endpoint (see cheat sheet). Default `both` (= execute + start) for Postman/curl; `--run both` runs `execute`. |
| `--target <t>` | (codegen only, **required**) `ts-zod`, `python-pydantic`, or `python-structures`. |
-| `--output-format <f>` | (build-output only) `schema` (default), `json`, or `python`. |
-| `--allow-signatures` | (validate + build routes) Tolerate unimplemented pipe signatures. Default strict. |
+| `--allow-signatures` | (validate only) Tolerate unimplemented pipe signatures. Default strict. |
| `--render <fmt>` | (validate only) Opt-in server-side view, e.g. `--render markdown` → response carries `rendered_markdown` (what a skill-driven validate sends). Omit for the lean structured body a hook reads. Repeatable. |
| `--inputs <path>` | Inputs JSON lives elsewhere, or you point at a `.mthds` file directly (file mode does not auto-detect inputs). execute/start only. |
-| `--pipe <pipe_code>` | The bundle has no `main_pipe`, or you want a different pipe. Used by execute/start (as `pipe_code`) and the build routes (qualified into `pipe_ref`); ignored by validate/resolve/codegen. |
+| `--pipe <pipe_code>` | The bundle has no `main_pipe`, or you want a different pipe. Used by execute/start (as `pipe_code`); ignored by validate/resolve/codegen. |
| `--callback-url <url>` | (start only) Webhook the async result is POSTed to. Repeatable. Falls back to `CALLBACK_URL` in env/`.env`. |
| `--base-url` / `--token` | Target server + bearer for `--run`/`--curl`. |
| `--name <folder>` | Override the per-bundle Postman subfolder name. |
@@ -197,9 +180,6 @@ Useful options (apply across modes):
- **"Validate / dry-run / just check it"** (no cost) → `--run --endpoint validate` (or `make bundle-validate`).
- **"Resolve it / get the crate / test /v1/resolve"** → `--run --endpoint resolve`.
- **"Generate types / codegen / zod schemas / pydantic models"** → `--run --endpoint codegen --target <t>` — ask which target if the user didn't say.
-- **"What inputs does this pipe need?" via the API** → `--run --endpoint build-inputs`.
-- **"What does the output look like?"** → `--run --endpoint build-output` (`--output-format json` for an example-shaped view).
-- **"Give me a runner script"** → `--run --endpoint build-runner`.
- **"Put it in Postman so I can click around"** → default mode (push), with the right `--endpoint`.
- **"Give me the curl"** → `--curl`. **Just checking resolution** → `--dry-run`.
@@ -207,7 +187,7 @@ Useful options (apply across modes):
- **Directory** → use `bundle.mthds` if present, else the single `*.mthds` in the dir. Multiple `.mthds` and no `bundle.mthds` is ambiguous → ask the user to pass the file directly. The sibling `inputs.json` is auto-detected. Every `*.mthds` in the directory is sent (main one first), so multi-file bundles resolve.
- **`.mthds` file** → used as-is; inputs only come from `--inputs`.
-- **`pipe_code`** (run routes) comes from the bundle's `main_pipe` (override with `--pipe`); the build routes send it qualified as `pipe_ref` (`domain.pipe_code`).
+- **`pipe_code`** (run routes) comes from the bundle's `main_pipe` (override with `--pipe`).
- **`inputs`** is the inputs.json content copied verbatim — exactly what the API receives, same as the CLI.
## Important: file/document inputs are out of scope
@@ -219,10 +199,10 @@ A related limit: the API receives only the inline bundle text, not the bundle di
## If something's missing
- **No `POSTMAN_API_KEY`** → the script exits asking you to `source ~/.zshenv`. Do that and retry. If still unset, the user needs to add it (Postman → Settings → API keys → Generate).
-- **No `main_pipe` and no `--pipe`** (execute/start/build routes) → the script exits; ask the user which pipe and pass `--pipe`.
+- **No `main_pipe` and no `--pipe`** (execute/start) → the script exits; ask the user which pipe and pass `--pipe`.
- **`codegen` without `--target`** → the script exits listing the three targets; ask the user which consumer they're generating for.
- **`start` selected via `--run`/`--curl` and no callback URL** → the script exits. Resolve from `--callback-url`, `CALLBACK_URL` in `.env`, or ask the user (e.g. a `https://webhook.site/...` endpoint). (The Postman push never hits this — it uses the `{{callback_url}}` variable and needs no resolution.)
-- **No `main_pipe`, validating/resolving** → fine, not an error: those endpoints need no pipe; validate just returns `graph_spec: null`. (On the build routes a missing `main_pipe` means `pipe_ref` cannot be defaulted — pass `--pipe`, or the server answers `422`.)
+- **No `main_pipe`, validating/resolving** → fine, not an error: those endpoints need no pipe; validate just returns `graph_spec: null`.
## Constants
diff --git a/api/.claude/skills/postman-bundle/scripts/build_postman_query.py b/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
index c111403f3..5b54e1fda 100644
--- a/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
+++ b/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
@@ -9,10 +9,9 @@ live "Pipelex FastAPI" Postman collection under ``Bundles/<bundle>/``.
Requests are generated per bundle (configurable via ``--endpoint``):
``Execute (sync)`` -> ``POST /v1/execute``, ``Start (async)`` ->
``POST /v1/start``, ``Validate (dry-run)`` -> ``POST /v1/validate``,
-``Resolve (crate)`` -> ``POST /v1/resolve``, ``Codegen (types)`` ->
-``POST /v1/codegen``, and the three bundle build routes ->
-``POST /v1/build/{inputs,output,runner}``. Only execute/start trigger
-inference — every other endpoint parses/loads/dry-runs with zero cost. All use
+``Resolve (crate)`` -> ``POST /v1/resolve``, and ``Codegen (types)`` ->
+``POST /v1/codegen``. Only execute/start trigger inference — every other
+endpoint parses/loads/dry-runs with zero cost. All use
the collection's ``{{base_url}}`` and inherit its ``{{auth_token}}`` bearer auth.
The async ``Start (async)`` body additionally carries ``callback_urls`` — the
@@ -59,9 +58,6 @@ TOP_FOLDER_NAME = "Bundles"
# "validate" -> {mthds_contents, allow_signatures [, mthds_sources, render]}
# "resolve" -> {files: [{content, source}]} — the crate envelope
# "codegen" -> the crate envelope plus {kind, target}
-# "build_inputs" -> the crate envelope plus {pipe_ref, format, explicit}
-# "build_output" -> the crate envelope plus {pipe_ref, format}
-# "build_runner" -> the crate envelope plus {pipe_ref, allow_signatures}
# Only "run"/"start" trigger inference; every other kind is a free
# parse/load/dry-run on the server.
ENDPOINTS: dict[str, tuple[str, list[str], str]] = {
@@ -70,22 +66,17 @@ ENDPOINTS: dict[str, tuple[str, list[str], str]] = {
"validate": ("Validate (dry-run)", ["v1", "validate"], "validate"),
"resolve": ("Resolve (crate)", ["v1", "resolve"], "resolve"),
"codegen": ("Codegen (types)", ["v1", "codegen"], "codegen"),
- "build-inputs": ("Build Inputs", ["v1", "build", "inputs"], "build_inputs"),
- "build-output": ("Build Output", ["v1", "build", "output"], "build_output"),
- "build-runner": ("Build Runner", ["v1", "build", "runner"], "build_runner"),
}
# Which endpoints need what. execute/start run a specific pipe with real inputs;
-# the build routes target a specific pipe but take no inputs; validate and the
-# crate routes (resolve/codegen) derive everything from the bundle text.
-PIPE_CODE_ENDPOINTS = {"execute", "start", "build-inputs", "build-output", "build-runner"}
+# validate and the crate routes (resolve/codegen) derive everything from the
+# bundle text.
+PIPE_CODE_ENDPOINTS = {"execute", "start"}
INPUTS_ENDPOINTS = {"execute", "start"}
-SIGNATURE_ENDPOINTS = {"validate", "build-runner"}
+SIGNATURE_ENDPOINTS = {"validate"}
CRATE_ENDPOINTS = {"resolve", "codegen"}
CODEGEN_TARGETS = ["ts-zod", "python-pydantic", "python-structures"]
-OUTPUT_FORMATS = ["schema", "json", "python"]
-INPUTS_FORMATS = ["json", "toml"]
def fail(msg: str) -> NoReturn:
@@ -333,7 +324,7 @@ def build_crate_body(
crate routes pair each content with its source in one ``{content, source}``
entry (the shape the codegen spec pins). No ``pipe_code``, no ``inputs`` —
the closure is the whole request. /codegen adds the two projection axes:
- ``kind`` (only ``types`` is served — input templates ride /build/inputs) and
+ ``kind`` (only ``types`` is served — an inputs template is projected client-side) and
``target`` (``ts-zod`` | ``python-pydantic`` | ``python-structures``).
"""
body: dict[str, Any] = {}
@@ -345,50 +336,6 @@ def build_crate_body(
return json.dumps(body, indent=2, ensure_ascii=False)
-def build_build_body(
- mthds_contents: list[str],
- mthds_sources: list[str],
- *,
- pipe_ref: str | None,
- allow_signatures: bool | None = None,
- output_format: str | None = None,
- inputs_format: str | None = None,
- explicit: bool | None = None,
-) -> str:
- """Body for /build/{inputs,output,runner} — the SAME ``files[]`` closure envelope as
- /resolve and /codegen, plus a pipe selector.
-
- ``pipe_ref`` is the qualified ``domain.pipe_code`` and is optional on the wire
- (it defaults to the closure's ``main_pipe``); it is sent explicitly here so the
- generated query is unambiguous and stays correct if the bundle later declares
- several main_pipes.
-
- Each route projects one artifact for that pipe — an inputs template (``format``:
- json | toml, plus ``explicit``; /build/inputs only), an output representation
- (``format``: schema | json | python; /build/output only), or a runner script. No
- ``inputs`` on the wire — nothing runs.
-
- ``format`` is a per-route axis with a different vocabulary on each, so the two are
- threaded separately (``inputs_format`` vs ``output_format``) and only ever one is set.
-
- ``allow_signatures`` is sent only for /build/runner: it parameterizes the dry-run
- sweep, and the static /build/{inputs,output} projections dropped that sweep.
- """
- body: dict[str, Any] = {}
- if pipe_ref:
- body["pipe_ref"] = pipe_ref
- if allow_signatures is not None:
- body["allow_signatures"] = allow_signatures
- if output_format:
- body["format"] = output_format
- if inputs_format:
- body["format"] = inputs_format
- if explicit is not None:
- body["explicit"] = explicit
- body["files"] = [{"content": content, "source": source} for content, source in zip(mthds_contents, mthds_sources)]
- return json.dumps(body, indent=2, ensure_ascii=False)
-
-
def build_request_item(name: str, path_parts: list[str], body_raw: str, description: str) -> dict[str, Any]:
return {
"name": name,
@@ -532,9 +479,8 @@ def main() -> None:
"--allow-signatures",
action="store_true",
help=(
- "(validate + build-runner) Tolerate unimplemented pipe signatures instead of rejecting the bundle. "
- "Parameterizes the dry-run sweep, so the static build-inputs/build-output projections do not take it. "
- "Default: strict."
+ "(validate only) Tolerate unimplemented pipe signatures instead of rejecting the bundle. "
+ "Parameterizes the dry-run sweep. Default: strict."
),
)
parser.add_argument(
@@ -546,29 +492,6 @@ def main() -> None:
"Required when --endpoint codegen."
),
)
- parser.add_argument(
- "--output-format",
- choices=OUTPUT_FORMATS,
- default="schema",
- help="(build-output only) Output representation format (default: schema). `python` returns source text in `output_python`.",
- )
- parser.add_argument(
- "--inputs-format",
- choices=INPUTS_FORMATS,
- default="json",
- help=(
- "(build-inputs only) Inputs template encoding (default: json). `json` returns a parsed object in `inputs`; "
- "`toml` returns raw text in `inputs_toml`, carrying the per-key `# concept:` comments JSON cannot."
- ),
- )
- parser.add_argument(
- "--explicit",
- action="store_true",
- help=(
- "(build-inputs only) Emit the ceremonial {concept, content} envelope for every input instead of the "
- "default light shape (a bare string for a Text input, and so on)."
- ),
- )
parser.add_argument(
"--render",
action="append",
@@ -587,9 +510,8 @@ def main() -> None:
help=(
"Which endpoint(s) to target (default: both = execute + start). Besides the run pair, "
"every choice is inference-free: 'validate' dry-runs the bundle, 'resolve' returns the "
- "normalized library crate, 'codegen' projects the crate into typed artifacts "
- "(needs --target), and 'build-inputs'/'build-output'/'build-runner' project one "
- "artifact for the requested pipe. For --run, 'both' runs execute (sync)."
+ "normalized library crate, and 'codegen' projects the crate into typed artifacts "
+ "(needs --target). For --run, 'both' runs execute (sync)."
),
)
parser.add_argument("--name", help="Override the per-bundle subfolder name (default: bundle domain or filename)")
@@ -624,7 +546,7 @@ def main() -> None:
needs_validate = any(ENDPOINTS[key][2] == "validate" for key in active_keys)
needs_crate = any(key in CRATE_ENDPOINTS for key in active_keys)
needs_signatures = any(key in SIGNATURE_ENDPOINTS for key in active_keys)
- # execute/start need pipe_code + inputs; the build routes need pipe_code only.
+ # execute/start need pipe_code + inputs.
needs_pipe = any(key in PIPE_CODE_ENDPOINTS for key in active_keys)
needs_inputs = any(key in INPUTS_ENDPOINTS for key in active_keys)
@@ -706,34 +628,6 @@ def main() -> None:
bodies["resolve"] = build_crate_body(mthds_contents, mthds_sources)
if "codegen" in active_keys:
bodies["codegen"] = build_crate_body(mthds_contents, mthds_sources, codegen_kind="types", codegen_target=args.target)
- # The build routes take the QUALIFIED pipe ref (`domain.pipe_code`).
- #
- # Only `main_pipe` is guaranteed to live in the MAIN file's domain, so only it may be qualified
- # with that domain. A `--pipe` override is passed through verbatim: in a multi-file closure the
- # user may well be naming a pipe in a sibling domain, and stapling the main file's domain onto it
- # would silently select the wrong pipe (or 422 on a ref that doesn't exist). A bare `--pipe` still
- # resolves — the engine's lookup matches a bare code too — and the user can always qualify it.
- if args.pipe:
- pipe_ref = args.pipe
- elif domain and main_pipe:
- pipe_ref = f"{domain}.{main_pipe}"
- else:
- pipe_ref = main_pipe
- if "build-inputs" in active_keys:
- bodies["build_inputs"] = build_build_body(
- mthds_contents,
- mthds_sources,
- pipe_ref=pipe_ref,
- inputs_format=args.inputs_format,
- explicit=args.explicit or None,
- )
- if "build-output" in active_keys:
- bodies["build_output"] = build_build_body(mthds_contents, mthds_sources, pipe_ref=pipe_ref, output_format=args.output_format)
- if "build-runner" in active_keys:
- # Only /build/runner still runs the dry-run sweep that `allow_signatures` parameterizes.
- bodies["build_runner"] = build_build_body(
- mthds_contents, mthds_sources, pipe_ref=pipe_ref, allow_signatures=args.allow_signatures
- )
print(f"Bundle: {main_file}")
print(f"mthds files: {', '.join(path.name for path in all_mthds)}")
@@ -753,8 +647,6 @@ def main() -> None:
print(f"crate: files[] envelope — sources: {', '.join(mthds_sources)} (no inference)")
if "codegen" in active_keys:
print(f"codegen: kind=types target={args.target}")
- if "build-output" in active_keys:
- print(f"format: {args.output_format}")
if local_url_warnings:
warn_target = "the API" if mode in ("run", "curl") else "Postman"
print("\nWARNING: inputs reference local (non-http) url(s) — file uploads are out of scope.")
@@ -836,40 +728,18 @@ def main() -> None:
"200 discriminated on `is_valid`: the canonical JSON crate on the valid arm, structured "
f"validation_errors[] on the invalid arm. {trailer}"
)
- if key == "codegen":
- return (
- f"Project the `{subfolder_name}` crate into typed artifacts (kind=types, target={args.target}) "
- "plus its codegen.lock. NO inference.\n\n"
- f"{source_lines}\n"
- "A client that writes the artifacts and lock verbatim reproduces a local `pipelex codegen types` "
- f"run byte-for-byte, so the offline `pipelex codegen check` passes. {trailer}"
- )
- # build-inputs / build-output / build-runner
- artifact = {
- "build-inputs": "the inputs JSON template",
- "build-output": f"the output representation (format={args.output_format})",
- "build-runner": "a runner script (plus the structures projection it imports)",
- }[key]
- if key == "build-runner":
- return (
- f"Generate {artifact} for pipe `{pipe_code}` of the `{subfolder_name}` bundle. Validates the "
- "bundle first (dry-run sweep scoped to the pipe, NO inference).\n\n"
- f"{source_lines}"
- f"- allow_signatures: {args.allow_signatures}\n\n"
- f"200 discriminated on `is_valid`. {trailer}"
- )
+ # codegen
return (
- f"Generate {artifact} for pipe `{pipe_code}` of the `{subfolder_name}` bundle. Static projection: "
- "resolves the closure and renders the pipe's declared IO — no dry-run sweep, NO inference.\n\n"
+ f"Project the `{subfolder_name}` crate into typed artifacts (kind=types, target={args.target}) "
+ "plus its codegen.lock. NO inference.\n\n"
f"{source_lines}\n"
- f"200 discriminated on `is_valid`. {trailer}"
+ "A client that writes the artifacts and lock verbatim reproduces a local `pipelex codegen types` "
+ f"run byte-for-byte, so the offline `pipelex codegen check` passes. {trailer}"
)
def display_name(key: str) -> str:
if key == "codegen":
return f"Codegen (types → {args.target})"
- if key == "build-output":
- return f"Build Output ({args.output_format})"
return ENDPOINTS[key][0]
requests = [build_request_item(display_name(key), ENDPOINTS[key][1], bodies[ENDPOINTS[key][2]], describe(key)) for key in selected]
diff --git a/api/CLAUDE.md b/api/CLAUDE.md
index 159c83abc..890203d99 100644
--- a/api/CLAUDE.md
+++ b/api/CLAUDE.md
@@ -1,6 +1,6 @@
# Pipelex API
-Pipelex API is the official FastAPI REST server for [Pipelex](https://github.com/Pipelex/pipelex). It wraps the Pipelex core library and exposes pipeline building, execution, and validation as HTTP endpoints. It is designed to be deployed via Docker and consumed by frontends, agents, and external services.
+Pipelex API is the official FastAPI REST server for [Pipelex](https://github.com/Pipelex/pipelex). It wraps the Pipelex core library and exposes pipeline execution, validation and method tooling as HTTP endpoints. It is designed to be deployed via Docker and consumed by frontends, agents, and external services.
## Where this server lives
@@ -35,8 +35,7 @@ pipelex_api/
codegen.py # POST /v1/codegen (crate → typed artifacts)
crate_ops.py # Closure resolution, the pipe selection chain, and the invalid-arm envelope the crate routes share
tools.py # POST /v1/lint, /v1/format (editor tooling)
- build/ # POST /v1/build/{inputs,output,runner}
- agent/ # POST /v1/build/{concept,pipe-spec}, GET /v1/models
+ agent/ # GET /v1/models
tests/
unit/ # Unit tests
scripts/
@@ -49,7 +48,7 @@ hatch_build.py # The metadata hook that writes the dependencies and
This server is the reference implementation of the [MTHDS Protocol](https://mthds.ai): `POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version` under the `/v1` base path, tagged `x-mthds-protocol: true` in the committed OpenAPI artifact (`docs/api-server/openapi/pipelex-api.openapi.yaml` at the repository root, regenerated via `make openapi-export`, drift-checked via `make openapi-check`). Contract nesting: MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex hosted API.
-**Only those five operations carry `x-mthds-protocol`** — the flag is how a conformance suite or a third-party runner extracts the portable subset of the artifact, so tagging a Pipelex route would misrepresent the standard. Everything else this server serves is a Pipelex API extension: `/resolve` + `/codegen`, `/pipe-io`, `/build/*`, and `/lint` + `/format`. Watch `/resolve` and `/codegen` in particular: they *look* protocol-shaped (they speak the `/validate` verdict discipline, and the crate `/resolve` emits is genuinely standard-owned — the MTHDS Library Crate Format, so its wire fields stay brand-neutral), but the routes are ours and the standard specifies no type projection at all. `tests/unit/test_openapi_contract.py` pins the tagged set exactly, in both directions.
+**Only those five operations carry `x-mthds-protocol`** — the flag is how a conformance suite or a third-party runner extracts the portable subset of the artifact, so tagging a Pipelex route would misrepresent the standard. Everything else this server serves is a Pipelex API extension: `/resolve` + `/codegen`, `/pipe-io`, and `/lint` + `/format`. Watch `/resolve` and `/codegen` in particular: they *look* protocol-shaped (they speak the `/validate` verdict discipline, and the crate `/resolve` emits is genuinely standard-owned — the MTHDS Library Crate Format, so its wire fields stay brand-neutral), but the routes are ours and the standard specifies no type projection at all. `tests/unit/test_openapi_contract.py` pins the tagged set exactly, in both directions.
pipelex's own models ride this wire (the run result, the bundle blueprint, the validation error vocabulary), so a library change can move the artifact with no change to the server's code. `openapi-check` is part of `make agent-check` for that reason: regenerate with `make openapi-export` and read the diff, which states what the change did to the server's callers.
diff --git a/api/Makefiles/Makefile.local.mk b/api/Makefiles/Makefile.local.mk
index 9ce26894d..1f259dba8 100644
--- a/api/Makefiles/Makefile.local.mk
+++ b/api/Makefiles/Makefile.local.mk
@@ -36,8 +36,8 @@ define HELP_LOCAL
make bundle-curl$(RESET) BUNDLE=<dir|.mthds>: Emit a ready-to-run curl command for the bundle.
make bundle-postman$(RESET) BUNDLE=<dir|.mthds>: Push request(s) into the live Pipelex FastAPI Postman collection.
make bundle-dry$(RESET) BUNDLE=<dir|.mthds>: Print the request body only — touch nothing.
- Optional: ENDPOINT=execute|start|validate|resolve|codegen|build-inputs|build-output|build-runner|both
- PIPE=<code> INPUTS=<path> NAME=<folder> ALLOW_SIGNATURES=1 TARGET=<codegen target> OUTPUT_FORMAT=schema|json|python
+ Optional: ENDPOINT=execute|start|validate|resolve|codegen|both
+ PIPE=<code> INPUTS=<path> NAME=<folder> ALLOW_SIGNATURES=1 TARGET=<codegen target>
CALLBACK_URL=<url> BASE_URL=<url> TOKEN=<bearer> ARGS=<extra>
(the async start endpoint needs CALLBACK_URL — set it here, or via CALLBACK_URL in .env)
@@ -208,14 +208,10 @@ tstop: temporal-stop
#
# Only execute/start trigger inference. bundle-validate (/v1/validate),
# bundle-resolve (/v1/resolve), and bundle-codegen (/v1/codegen) are free —
-# parse/load/dry-run only — so they hardcode --run. The build routes
-# (/v1/build/{inputs,output,runner}) ride the generic targets via ENDPOINT=,
-# e.g. `make bundle-run ENDPOINT=build-inputs` or
-# `make bundle-postman ENDPOINT=build-output OUTPUT_FORMAT=json`.
+# parse/load/dry-run only — so they hardcode --run.
#
# Optional pass-throughs: ENDPOINT, PIPE, INPUTS, NAME, ALLOW_SIGNATURES,
-# TARGET (codegen only — required), OUTPUT_FORMAT (build-output only),
-# RENDER (validate only — e.g. RENDER=markdown reproduces a skill-driven
+# TARGET (codegen only — required), RENDER (validate only — e.g. RENDER=markdown reproduces a skill-driven
# validate, response carries rendered_markdown), CALLBACK_URL (all modes);
# BASE_URL, TOKEN (run/curl); ARGS for anything else.
# The async start endpoint requires CALLBACK_URL — pass it here, or set
@@ -223,7 +219,7 @@ tstop: temporal-stop
# POSTMAN_API_KEY in the environment (it lives in ~/.zshenv, so any zsh-launched
# make has it).
BUNDLE_SCRIPT := .claude/skills/postman-bundle/scripts/build_postman_query.py
-BUNDLE_OPTS = $(if $(ENDPOINT),--endpoint $(ENDPOINT)) $(if $(PIPE),--pipe $(PIPE)) $(if $(INPUTS),--inputs $(INPUTS)) $(if $(NAME),--name $(NAME)) $(if $(ALLOW_SIGNATURES),--allow-signatures) $(if $(TARGET),--target $(TARGET)) $(if $(OUTPUT_FORMAT),--output-format $(OUTPUT_FORMAT)) $(if $(RENDER),--render $(RENDER)) $(if $(CALLBACK_URL),--callback-url '$(CALLBACK_URL)') $(ARGS)
+BUNDLE_OPTS = $(if $(ENDPOINT),--endpoint $(ENDPOINT)) $(if $(PIPE),--pipe $(PIPE)) $(if $(INPUTS),--inputs $(INPUTS)) $(if $(NAME),--name $(NAME)) $(if $(ALLOW_SIGNATURES),--allow-signatures) $(if $(TARGET),--target $(TARGET)) $(if $(RENDER),--render $(RENDER)) $(if $(CALLBACK_URL),--callback-url '$(CALLBACK_URL)') $(ARGS)
BUNDLE_RUN_OPTS = $(if $(BASE_URL),--base-url $(BASE_URL)) $(if $(TOKEN),--token $(TOKEN))
check-bundle-arg:
diff --git a/api/README.md b/api/README.md
index 3f7d374f9..0d49a53d7 100644
--- a/api/README.md
+++ b/api/README.md
@@ -3,7 +3,7 @@
<h2 align="center">Pipelex API</h2>
-The official REST API server for building and executing Pipelex pipelines. Deploy your pipelines as HTTP endpoints and integrate them into any application or workflow.
+The official REST API server for validating and executing Pipelex pipelines. Deploy your pipelines as HTTP endpoints and integrate them into any application or workflow.
<div>
<a href="https://docs.pipelex.com/latest/api-server/"><strong>API Documentation</strong></a> -
@@ -37,7 +37,7 @@ The official REST API server for building and executing Pipelex pipelines. Deplo
The **Pipelex API Server** is a FastAPI-based REST API that allows you to execute [Pipelex](https://github.com/Pipelex/pipelex) pipelines via HTTP requests. Deploy your pipelines as HTTP endpoints and integrate them into any application or workflow.
-It is the source-available reference implementation of the **[MTHDS Protocol](https://mthds.ai)** — the minimal HTTP contract every MTHDS runner implements (`POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version`). The contracts nest: **MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API**. This server adds the build tooling extensions (`/build/*`) on top of the protocol; the hosted API at `api.pipelex.com/v1` adds durable runs, the method catalog, and account management on top of this server — same shapes throughout. All routes live under the `/v1` base path; the committed contract is [`pipelex-api.openapi.yaml`](https://github.com/Pipelex/pipelex/blob/main/docs/api-server/openapi/pipelex-api.openapi.yaml).
+It is the source-available reference implementation of the **[MTHDS Protocol](https://mthds.ai)** — the minimal HTTP contract every MTHDS runner implements (`POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version`). The contracts nest: **MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API**. This server adds the tooling extensions (`/resolve`, `/codegen`, `/pipe-io`, `/lint`, `/format`) on top of the protocol; the hosted API at `api.pipelex.com/v1` adds durable runs, the method catalog, and account management on top of this server — same shapes throughout. All routes live under the `/v1` base path; the committed contract is [`pipelex-api.openapi.yaml`](https://github.com/Pipelex/pipelex/blob/main/docs/api-server/openapi/pipelex-api.openapi.yaml).
# 🚀 Quick Start with Docker
@@ -119,7 +119,8 @@ The full reference for this API server is part of the Pipelex documentation, und
- [Overview](https://docs.pipelex.com/latest/api-server/) — endpoints, authentication, deployment
- [Pipe Run](https://docs.pipelex.com/latest/api-server/pipe-run/) — `/execute`, `/start`, every input shape
- [Pipe Validate](https://docs.pipelex.com/latest/api-server/pipe-validate/) — `/validate`
-- [Pipe Builder](https://docs.pipelex.com/latest/api-server/pipe-builder/) — `/build/inputs`, `/build/output`, `/build/runner`
+- [Pipe I/O](https://docs.pipelex.com/latest/api-server/pipe-io/) — `/pipe-io`
+- [Resolve & Codegen](https://docs.pipelex.com/latest/api-server/codegen/) — `/resolve`, `/codegen`
- [Configuration](https://docs.pipelex.com/latest/api-server/configuration/) — env vars, mounting your own `.pipelex/` config
For broader Pipelex documentation (MTHDS language, concepts, pipe types, the Gateway): **[https://docs.pipelex.com/](https://docs.pipelex.com/)**
diff --git a/api/pipelex_api/limits.py b/api/pipelex_api/limits.py
index 721bee1bc..a92c2a8b6 100644
--- a/api/pipelex_api/limits.py
+++ b/api/pipelex_api/limits.py
@@ -24,7 +24,6 @@ MAX_METHOD_REF_LEN = 512 # `method_ref` selector strings; a fixed schema bound,
MAX_JSON_NESTING_DEPTH = 128
DEFAULT_MAX_CALLBACK_URLS = 5
DEFAULT_MAX_CALLBACK_URL_LEN = 2048
-DEFAULT_MAX_AGENT_SPEC_KIB = 256 # 256 KiB for JSON concept/pipe specs
DEFAULT_MAX_BUNDLE_FILES = 128 # entries in a materialized method bundle (.mthds + .py + requirements.txt)
DEFAULT_MAX_BUNDLE_TOTAL_KIB = 8 * 1024 # 8 MiB decompressed across the whole bundle (zip-bomb guard)
DEFAULT_MAX_METHOD_CACHE_CLONES = 64 # cached method-package clones (one per resolved commit SHA)
@@ -57,8 +56,6 @@ MAX_PIPE_CODE_LEN = _read_positive_int("MAX_PIPE_CODE_LEN", DEFAULT_MAX_PIPE_COD
MAX_CALLBACK_URLS = _read_positive_int("MAX_CALLBACK_URLS", DEFAULT_MAX_CALLBACK_URLS)
MAX_CALLBACK_URL_LEN = _read_positive_int("MAX_CALLBACK_URL_LEN", DEFAULT_MAX_CALLBACK_URL_LEN)
-MAX_AGENT_SPEC_BYTES = _read_positive_int("MAX_AGENT_SPEC_KIB", DEFAULT_MAX_AGENT_SPEC_KIB) * 1024
-
MAX_BUNDLE_FILES = _read_positive_int("MAX_BUNDLE_FILES", DEFAULT_MAX_BUNDLE_FILES)
MAX_BUNDLE_TOTAL_BYTES = _read_positive_int("MAX_BUNDLE_TOTAL_KIB", DEFAULT_MAX_BUNDLE_TOTAL_KIB) * 1024
diff --git a/api/pipelex_api/main.py b/api/pipelex_api/main.py
index d8dd65cd2..4de7c0936 100644
--- a/api/pipelex_api/main.py
+++ b/api/pipelex_api/main.py
@@ -150,7 +150,7 @@ fastapi_app = PipelexFastAPI(
f"This server implements the [MTHDS Protocol](https://mthds.ai) v{PROTOCOL_VERSION} "
"(`POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version` — "
"marked `x-mthds-protocol: true`) plus the Pipelex API extensions: resolve and codegen "
- "(`/resolve`, `/codegen`), build tooling (`/build/*`), and editor tooling (`/lint`, `/format`). "
+ "(`/resolve`, `/codegen`), the pipe I/O route (`/pipe-io`), and editor tooling (`/lint`, `/format`). "
"Contract layering: MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API. "
"All endpoints are served under the `/v1` base path; "
"every error is an RFC 7807 `application/problem+json` problem document, documented per "
diff --git a/api/pipelex_api/method_source.py b/api/pipelex_api/method_source.py
index e74be9d8a..4e326ce85 100644
--- a/api/pipelex_api/method_source.py
+++ b/api/pipelex_api/method_source.py
@@ -13,10 +13,10 @@ manifest-identity package location, the bounds, and the structures check all liv
`mthds_sources`, so diagnostics carry true per-file labels), and only the non-`.mthds`
files are materialized into a temporary `library_dirs` entry — exactly the split the
method-bundle transport uses, so a fetched package runs the same proven path.
-- The tooling routes (`/resolve`, `/codegen`, `/build/*`) use
+- The tooling routes (`/resolve`, `/codegen`, `/pipe-io`) use
:func:`fetch_method_mthds_files`: only the `.mthds` files, as `files[]` items, paired with
- the manifest's `main_pipe` so the per-pipe projections default their selector exactly as a
- run does. No Python ever loads there, so the execution-locus gate does not apply.
+ the manifest's `main_pipe` so the per-pipe route defaults its selector exactly as a run
+ does. No Python ever loads there, so the execution-locus gate does not apply.
The security gate (packaging invariant 7 — execution locus decides): `.mthds` content is
data, always acceptable. On a deployment that is NOT sandbox-hosted, a fetched package
diff --git a/api/pipelex_api/openapi_responses.py b/api/pipelex_api/openapi_responses.py
index 9cb892083..136534754 100644
--- a/api/pipelex_api/openapi_responses.py
+++ b/api/pipelex_api/openapi_responses.py
@@ -82,7 +82,7 @@ class ProblemDocument(BaseModel):
description=(
"Structured per-error diagnostics, carried by a `ValidateBundleError`. Each item may carry a `suggested_fix`. "
"Retained under `ERROR_DISCLOSURE=strict` — it describes the caller's own bundle, not server internals. "
- "On `/validate`, `/resolve`, `/codegen` and `/build/*` an invalid bundle is a **200** verdict instead, so the "
+ "On `/validate`, `/resolve`, `/codegen` and `/pipe-io` an invalid bundle is a **200** verdict instead, so the "
"items ride the response body there rather than a problem document."
),
)
diff --git a/api/pipelex_api/routes/pipelex/__init__.py b/api/pipelex_api/routes/pipelex/__init__.py
index 69de2873d..6c0426ba4 100644
--- a/api/pipelex_api/routes/pipelex/__init__.py
+++ b/api/pipelex_api/routes/pipelex/__init__.py
@@ -1,7 +1,6 @@
from fastapi import APIRouter
from .agent import router as agent_router
-from .build import router as build_router
from .codegen import router as codegen_router
from .pipe_io import router as pipe_io_router
from .pipeline import router as pipeline_router
@@ -11,7 +10,6 @@ from .validate import router as validate_router
router = APIRouter()
-router.include_router(build_router)
router.include_router(pipeline_router)
router.include_router(validate_router)
router.include_router(resolve_router)
diff --git a/api/pipelex_api/routes/pipelex/agent/__init__.py b/api/pipelex_api/routes/pipelex/agent/__init__.py
index c4d4e56de..9d95a11ec 100644
--- a/api/pipelex_api/routes/pipelex/agent/__init__.py
+++ b/api/pipelex_api/routes/pipelex/agent/__init__.py
@@ -1,11 +1,7 @@
from fastapi import APIRouter
-from .concept import router as concept_router
from .models import router as models_router
-from .pipe_spec import router as pipe_spec_router
router = APIRouter()
router.include_router(models_router)
-router.include_router(concept_router)
-router.include_router(pipe_spec_router)
diff --git a/api/pipelex_api/routes/pipelex/agent/concept.py b/api/pipelex_api/routes/pipelex/agent/concept.py
deleted file mode 100644
index 96bac9fe9..000000000
--- a/api/pipelex_api/routes/pipelex/agent/concept.py
+++ /dev/null
@@ -1,60 +0,0 @@
-"""Concept endpoint — convert JSON concept spec to TOML."""
-
-import json
-from typing import Any
-
-from fastapi import APIRouter
-from pipelex.builder.operations.concept_ops import concept_spec_to_toml, parse_concept_spec
-from pydantic import BaseModel, Field, ValidationError, field_validator
-
-from pipelex_api.errors import raise_validation_error
-from pipelex_api.json_body import JsonBodyRoute
-from pipelex_api.limits import MAX_AGENT_SPEC_BYTES
-
-router = APIRouter(tags=["agent"], route_class=JsonBodyRoute)
-
-
-class BuildConceptRequest(BaseModel):
- spec: dict[str, Any] = Field(..., description="JSON concept specification.")
-
- @field_validator("spec")
- @classmethod
- def _bound_spec_size(cls, value: dict[str, Any]) -> dict[str, Any]:
- if len(json.dumps(value).encode("utf-8")) > MAX_AGENT_SPEC_BYTES:
- msg = f"spec exceeds {MAX_AGENT_SPEC_BYTES // 1024} KiB limit"
- raise ValueError(msg)
- return value
-
-
-class BuildConceptResponse(BaseModel):
- success: bool = Field(default=True, description="Whether the operation was successful")
- concept_code: str = Field(..., description="The concept code that was generated")
- toml: str = Field(..., description="Generated TOML content for the concept")
-
-
[email protected]("/build/concept")
-async def build_concept(request_data: BuildConceptRequest) -> BuildConceptResponse:
- """Convert a JSON concept spec to TOML format.
-
- A malformed spec surfaces as a Pydantic `ValidationError` — an API-owned
- 422. Pipelex domain failures propagate untouched to the global
- `PipelexError` handler in `pipelex_api.exception_handlers`.
-
- A malformed shape — a non-mapping `structure` (`{"structure": "string"}`)
- or a `structure` field that is neither a string nor a mapping
- (`{"structure": {"f": 42}}`) — is refused by `parse_concept_spec` itself
- with a typed `ConceptSpecError`, which that same handler answers. Nothing
- broader is caught here: a bare exception from pipelex is the signal of a
- real programming bug, and a route-level catch would mask it.
- """
- try:
- concept_spec = parse_concept_spec(request_data.spec)
- toml_content = concept_spec_to_toml(concept_spec)
-
- return BuildConceptResponse(
- success=True,
- concept_code=concept_spec.concept_code,
- toml=toml_content,
- )
- except ValidationError as exc:
- raise_validation_error(message=str(exc))
diff --git a/api/pipelex_api/routes/pipelex/agent/pipe_spec.py b/api/pipelex_api/routes/pipelex/agent/pipe_spec.py
deleted file mode 100644
index a57e7d731..000000000
--- a/api/pipelex_api/routes/pipelex/agent/pipe_spec.py
+++ /dev/null
@@ -1,59 +0,0 @@
-"""Pipe spec endpoint — convert JSON pipe spec to TOML."""
-
-import json
-from typing import Any
-
-from fastapi import APIRouter
-from pipelex.builder.operations.pipe_ops import parse_pipe_spec, pipe_spec_to_toml
-from pydantic import BaseModel, Field, ValidationError, field_validator
-
-from pipelex_api.errors import raise_validation_error
-from pipelex_api.json_body import JsonBodyRoute
-from pipelex_api.limits import MAX_AGENT_SPEC_BYTES
-
-router = APIRouter(tags=["agent"], route_class=JsonBodyRoute)
-
-
-class BuildPipeSpecRequest(BaseModel):
- pipe_type: str = Field(..., min_length=1, max_length=128, description="The pipe type (e.g. PipeLLM, PipeSequence, etc.).")
- spec: dict[str, Any] = Field(..., description="JSON pipe specification.")
-
- @field_validator("spec")
- @classmethod
- def _bound_spec_size(cls, value: dict[str, Any]) -> dict[str, Any]:
- if len(json.dumps(value).encode("utf-8")) > MAX_AGENT_SPEC_BYTES:
- msg = f"spec exceeds {MAX_AGENT_SPEC_BYTES // 1024} KiB limit"
- raise ValueError(msg)
- return value
-
-
-class BuildPipeSpecResponse(BaseModel):
- success: bool = Field(default=True, description="Whether the operation was successful")
- pipe_code: str = Field(..., description="The pipe code that was generated")
- pipe_type: str = Field(..., description="The pipe type")
- toml: str = Field(..., description="Generated TOML content for the pipe")
-
-
[email protected]("/build/pipe-spec")
-async def build_pipe_spec(request_data: BuildPipeSpecRequest) -> BuildPipeSpecResponse:
- """Convert a JSON pipe spec to TOML format.
-
- Two caller-mistake paths surface as a 422 here: a `ValidationError` from
- Pydantic when the spec shape doesn't match the chosen pipe type, and a
- `ValueError` from `parse_pipe_spec` when `pipe_type` is not one of the
- known pipe types (documented in `parse_pipe_spec`'s docstring; raised at
- exactly one site). Pipelex domain failures propagate untouched to the
- global `PipelexError` handler in `pipelex_api.exception_handlers`.
- """
- try:
- pipe_spec = parse_pipe_spec(request_data.spec, pipe_type=request_data.pipe_type)
- toml_content = pipe_spec_to_toml(pipe_spec)
-
- return BuildPipeSpecResponse(
- success=True,
- pipe_code=pipe_spec.pipe_code,
- pipe_type=request_data.pipe_type,
- toml=toml_content,
- )
- except (ValidationError, ValueError) as exc:
- raise_validation_error(message=str(exc))
diff --git a/api/pipelex_api/routes/pipelex/build/__init__.py b/api/pipelex_api/routes/pipelex/build/__init__.py
deleted file mode 100644
index 979ea6b5f..000000000
--- a/api/pipelex_api/routes/pipelex/build/__init__.py
+++ /dev/null
@@ -1,11 +0,0 @@
-from fastapi import APIRouter
-
-from .inputs import router as inputs_router
-from .output import router as output_router
-from .runner import router as runner_router
-
-router = APIRouter()
-
-router.include_router(inputs_router)
-router.include_router(output_router)
-router.include_router(runner_router)
diff --git a/api/pipelex_api/routes/pipelex/build/inputs.py b/api/pipelex_api/routes/pipelex/build/inputs.py
deleted file mode 100644
index 04d9c664a..000000000
--- a/api/pipelex_api/routes/pipelex/build/inputs.py
+++ /dev/null
@@ -1,192 +0,0 @@
-import json
-from typing import Annotated, Any, Literal, Self, Union
-
-from fastapi import APIRouter
-from fastapi.responses import JSONResponse
-from pipelex.core.pipes.inputs.exceptions import NoInputsRequiredError
-from pipelex.pipe_machinery.rendering.input_renderer import InputsTemplateFormat, render_inputs, render_inputs_toml
-from pipelex.pipeline.exceptions import ValidateBundleError
-from pydantic import BaseModel, Field, model_validator
-
-from pipelex_api.json_body import JsonBodyRoute
-from pipelex_api.openapi_responses import PROBLEM_404_METHOD_PACKAGE, PROBLEM_501_METHOD_REF
-from pipelex_api.routes.pipelex.crate_ops import (
- CrateInvalidReport,
- RequestedPipe,
- invalid_crate_report_response,
- resolve_requested_crate,
- resolve_requested_pipe,
- teardown_current_library,
-)
-from pipelex_api.schemas.models import MthdsPipeRequest
-
-router = APIRouter(tags=["build"], route_class=JsonBodyRoute)
-
-INPUTS_GENERATED_MESSAGE = "Inputs template generated successfully"
-NO_INPUTS_MESSAGE = "This pipe declares no inputs — the template is empty."
-
-
-class BuildInputsRequest(MthdsPipeRequest):
- """The inputs-template request: the shared closure + pipe selectors, plus the two rendering axes.
-
- Both axes mirror `pipelex codegen inputs` exactly — `--format` and `--explicit` — and every
- combination of them is served, as on the CLI.
- """
-
- format: InputsTemplateFormat = Field(
- default=InputsTemplateFormat.JSON,
- description="Template encoding. `json` returns the parsed template in `inputs`; `toml` returns the raw text in `inputs_toml`.",
- )
- explicit: bool = Field(
- default=False,
- description=(
- "When true, emit the ceremonial `{concept, content}` envelope for every input. Defaults to false — the light, "
- "signature-driven shape that smart inputs accepts (a bare string for a Text input, and so on)."
- ),
- )
-
-
-class BuildInputsValidReport(BaseModel):
- """The 200 **valid** arm: the example inputs template for the requested pipe.
-
- The template rides **one of two fields, chosen by `format`** — `inputs` (parsed object) for
- `json`, `inputs_toml` (raw text) for `toml`. TOML cannot be carried as a parsed object without
- losing what makes it worth asking for (its concept comments and key order), and the JSON case
- must stay a real object, since that is what the deploy dialog and the SDKs consume. So the two
- are separate, honestly-typed fields, and the unused one is omitted from the body entirely.
-
- A pipe that declares no inputs is a *valid* verdict, not an error (the CLI likewise exits 0):
- the template is simply empty, and `message` says so.
- """
-
- is_valid: Literal[True] = True
- pipe_ref: str = Field(..., description="The qualified pipe the template was generated for — the resolved selector.")
- requested_pipe_ref: str | None = Field(
- default=None,
- description=(
- "The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's "
- "`main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`."
- ),
- )
- format: InputsTemplateFormat = Field(..., description="The template encoding (echo of the request).")
- explicit: bool = Field(..., description="Whether the ceremonial envelope shape was emitted (echo of the request).")
- inputs: dict[str, Any] | None = Field(
- default=None,
- description="The parsed inputs template — present exactly when `format` is `json`.",
- )
- inputs_toml: str | None = Field(
- default=None,
- description="The inputs template as TOML text — present exactly when `format` is `toml`.",
- )
- message: str = Field(default=INPUTS_GENERATED_MESSAGE, description="Status message")
-
- @model_validator(mode="after")
- def _template_field_matches_format(self) -> Self:
- # The invariant the two-field shape exists to keep: the caller can read the field its own
- # `format` names, without probing. A route bug that set the other one would be a silent
- # contract break, so it fails loudly here instead.
- match self.format:
- case InputsTemplateFormat.JSON:
- if self.inputs is None or self.inputs_toml is not None:
- msg = "format='json' must carry `inputs` and no `inputs_toml`"
- raise ValueError(msg)
- case InputsTemplateFormat.TOML:
- if self.inputs_toml is None or self.inputs is not None:
- msg = "format='toml' must carry `inputs_toml` and no `inputs`"
- raise ValueError(msg)
- return self
-
-
-# Discriminated 200 response union: the `/validate` discipline — the verdict rides `is_valid`, never
-# the HTTP status.
-BuildInputsResponse = Annotated[Union[BuildInputsValidReport, CrateInvalidReport], Field(discriminator="is_valid")]
-
-
-def _render_report(*, requested: BuildInputsRequest, requested_pipe: RequestedPipe) -> BuildInputsValidReport:
- """Render the requested pipe's declared inputs into the valid arm, on the `format` axis.
-
- A pipe with no inputs raises `NoInputsRequiredError` out of the engine renderers. That is not a
- failure — it is the honest answer "run this with nothing" — so it lands on the valid arm as an
- empty template, mirroring the CLI's exit-0.
- """
- inputs: dict[str, Any] | None = None
- inputs_toml: str | None = None
- message: str = INPUTS_GENERATED_MESSAGE
- match requested.format:
- case InputsTemplateFormat.JSON:
- try:
- inputs = json.loads(render_inputs(requested_pipe.pipe, explicit=requested.explicit))
- except NoInputsRequiredError:
- inputs = {}
- message = NO_INPUTS_MESSAGE
- case InputsTemplateFormat.TOML:
- try:
- inputs_toml = render_inputs_toml(requested_pipe.pipe, explicit=requested.explicit)
- except NoInputsRequiredError:
- inputs_toml = ""
- message = NO_INPUTS_MESSAGE
- return BuildInputsValidReport(
- pipe_ref=requested_pipe.ref,
- requested_pipe_ref=requested.pipe_ref,
- format=requested.format,
- explicit=requested.explicit,
- inputs=inputs,
- inputs_toml=inputs_toml,
- message=message,
- )
-
-
- "/build/inputs",
- response_model=BuildInputsResponse,
- # On top of the composite router's shared 401/413/422/500: the `method_ref` closure selector the
- # envelope accepts but no server-side method registry resolves yet (shared with /resolve, /codegen).
- responses={404: PROBLEM_404_METHOD_PACKAGE, 501: PROBLEM_501_METHOD_REF},
-)
-async def build_inputs(request_data: BuildInputsRequest) -> JSONResponse:
- """Generate an example inputs template for a pipe (the inputs projection, per pipe).
-
- Rides the **same static core** as `POST /resolve` and `POST /codegen`: the closure is resolved to
- its normalized crate, the requested pipe is read live from the library that leaves loaded, and its
- *declared* inputs are rendered — the exact projection `pipelex codegen inputs` writes, on both of
- its axes (`format`, `explicit`).
-
- Static is the point: a template is a read of the pipe's declared IO, so there is **no dry-run
- sweep** here (that is `/validate`'s vocabulary, and `/build/runner`'s need). A valid verdict says
- the closure is structurally sound and the template matches what the pipe declares — it is *not* a
- promise the pipe runs. Ask `/validate` for that.
-
- This projection is deliberately **not** a `/codegen` kind: the templates are user-editable
- scaffolds, never stamped or locked, so they cannot ride the trust chain `/codegen`'s valid arm
- promises (see `CodegenRouteKind`).
-
- Response contract (the `/validate` discipline):
-
- - **Valid verdict (200, `is_valid: true`):** the template, in `inputs` or `inputs_toml` per `format`.
- - **Invalid verdict (200, `is_valid: false`):** the closure could not be parsed, loaded, or
- validated — `validation_errors[]` from pipelex's one shared builder; no template exists.
- - **No verdict (non-2xx):** a selection refusal — an unknown pipe ref, or an omitted `pipe_ref` that
- nothing defaults (no fetched-manifest `main_pipe`, and a closure declaring no — or several —
- `main_pipe`) — is an input 422 whose `error_type` is `EntryPipeNotFoundError` or
- `EntryPipeAmbiguousError`; a malformed closure selector is a request-shape 422 problem+json; a
- registry-form `method_ref` is a 501 until server-side method-registry resolution exists.
-
- An omitted `pipe_ref` defaults the way a run by address does: to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's own declared `main_pipe`.
- """
- try:
- resolved = resolve_requested_crate(request_data)
- except ValidateBundleError as validate_error:
- return invalid_crate_report_response(validate_error.to_error_report())
- try:
- requested_pipe = resolve_requested_pipe(
- resolved.crate,
- pipe_ref=request_data.pipe_ref,
- manifest_main_pipe=resolved.manifest_main_pipe,
- )
- report = _render_report(requested=request_data, requested_pipe=requested_pipe)
- # exclude_none drops the template field the `format` did not select (and an absent
- # `requested_pipe_ref`), so exactly the fields the caller's own request implies are present.
- return JSONResponse(content=report.model_dump(mode="json", by_alias=True, exclude_none=True))
- finally:
- teardown_current_library()
diff --git a/api/pipelex_api/routes/pipelex/build/output.py b/api/pipelex_api/routes/pipelex/build/output.py
deleted file mode 100644
index 1bc2c4007..000000000
--- a/api/pipelex_api/routes/pipelex/build/output.py
+++ /dev/null
@@ -1,163 +0,0 @@
-import json
-from typing import Annotated, Any, Literal, Self, Union
-
-from fastapi import APIRouter
-from fastapi.responses import JSONResponse
-from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
-from pipelex.pipe_machinery.rendering.output_renderer import render_output
-from pipelex.pipeline.exceptions import ValidateBundleError
-from pydantic import BaseModel, Field, model_validator
-
-from pipelex_api.errors import raise_validation_error
-from pipelex_api.json_body import JsonBodyRoute
-from pipelex_api.openapi_responses import PROBLEM_404_METHOD_PACKAGE, PROBLEM_501_METHOD_REF
-from pipelex_api.routes.pipelex.crate_ops import (
- CrateInvalidReport,
- RequestedPipe,
- invalid_crate_report_response,
- resolve_requested_crate,
- resolve_requested_pipe,
- teardown_current_library,
-)
-from pipelex_api.schemas.models import MthdsPipeRequest
-
-router = APIRouter(tags=["build"], route_class=JsonBodyRoute)
-
-
-class BuildOutputRequest(MthdsPipeRequest):
- """The output-representation request: the shared closure + pipe selectors, plus the format axis."""
-
- format: ConceptRepresentationFormat = Field(
- default=ConceptRepresentationFormat.SCHEMA,
- description=(
- "Representation to render. `schema` (JSON Schema) and `json` (example value) return a parsed object in "
- "`output`; `python` returns Python source in `output_python`."
- ),
- )
-
-
-class BuildOutputValidReport(BaseModel):
- """The 200 **valid** arm: the example output representation for the requested pipe.
-
- The representation rides **one of two fields, chosen by `format`**, for the same reason
- `/build/inputs` splits `inputs` / `inputs_toml`: `schema` and `json` are objects, `python` is
- source text. (Before this split the route parsed *every* format as JSON, so `format=python` was a
- hard 500 — it fed Python source to `json.loads`.)
- """
-
- is_valid: Literal[True] = True
- pipe_ref: str = Field(..., description="The qualified pipe the representation was generated for — the resolved selector.")
- requested_pipe_ref: str | None = Field(
- default=None,
- description=(
- "The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's "
- "`main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`."
- ),
- )
- format: ConceptRepresentationFormat = Field(..., description="The representation format (echo of the request).")
- output: dict[str, Any] | None = Field(
- default=None,
- description="The parsed output representation — present exactly when `format` is `schema` or `json`.",
- )
- output_python: str | None = Field(
- default=None,
- description="The output representation as Python source — present exactly when `format` is `python`.",
- )
- message: str = Field(default="Output representation generated successfully", description="Status message")
-
- @model_validator(mode="after")
- def _representation_field_matches_format(self) -> Self:
- # Same invariant as `/build/inputs`: the caller reads the field its own `format` names.
- match self.format:
- case ConceptRepresentationFormat.SCHEMA | ConceptRepresentationFormat.JSON:
- if self.output is None or self.output_python is not None:
- msg = "format='schema'/'json' must carry `output` and no `output_python`"
- raise ValueError(msg)
- case ConceptRepresentationFormat.PYTHON:
- if self.output_python is None or self.output is not None:
- msg = "format='python' must carry `output_python` and no `output`"
- raise ValueError(msg)
- return self
-
-
-# Discriminated 200 response union: the `/validate` discipline — the verdict rides `is_valid`, never
-# the HTTP status.
-BuildOutputResponse = Annotated[Union[BuildOutputValidReport, CrateInvalidReport], Field(discriminator="is_valid")]
-
-
-def _render_report(*, requested: BuildOutputRequest, requested_pipe: RequestedPipe) -> BuildOutputValidReport:
- """Render the requested pipe's declared output into the valid arm, on the `format` axis.
-
- `render_output` raises a bare `ValueError` (documented in its docstring) when the pipe's output is
- `native.Anything` and no concrete option can be determined — a fact about the *requested pipe*,
- not about the closure, so it is a no-verdict 422 rather than an invalid-crate verdict.
- """
- output: dict[str, Any] | None = None
- output_python: str | None = None
- try:
- rendered = render_output(requested_pipe.pipe, output_format=requested.format)
- except ValueError as exc:
- raise_validation_error(f"Cannot render the output representation of pipe '{requested_pipe.ref}': {exc}")
- match requested.format:
- case ConceptRepresentationFormat.SCHEMA | ConceptRepresentationFormat.JSON:
- output = json.loads(rendered)
- case ConceptRepresentationFormat.PYTHON:
- output_python = rendered
- return BuildOutputValidReport(
- pipe_ref=requested_pipe.ref,
- requested_pipe_ref=requested.pipe_ref,
- format=requested.format,
- output=output,
- output_python=output_python,
- )
-
-
- "/build/output",
- response_model=BuildOutputResponse,
- # On top of the composite router's shared 401/413/422/500: the `method_ref` closure selector the
- # envelope accepts but no server-side method registry resolves yet (shared with /resolve, /codegen).
- responses={404: PROBLEM_404_METHOD_PACKAGE, 501: PROBLEM_501_METHOD_REF},
-)
-async def build_output(request_data: BuildOutputRequest) -> JSONResponse:
- """Generate an example output representation for a pipe (the output projection, per pipe).
-
- Rides the **same static core** as `POST /resolve`, `POST /codegen` and `POST /build/inputs`: the
- closure is resolved to its normalized crate, the requested pipe is read live from the library that
- leaves loaded, and its *declared* output is rendered.
-
- Static is the point: the representation is a read of the pipe's declared IO, so there is **no
- dry-run sweep** here. A valid verdict says the closure is structurally sound and the shape matches
- what the pipe declares — it is *not* a promise the pipe runs. Ask `/validate` for that.
-
- Response contract (the `/validate` discipline):
-
- - **Valid verdict (200, `is_valid: true`):** the representation, in `output` or `output_python` per `format`.
- - **Invalid verdict (200, `is_valid: false`):** the closure could not be parsed, loaded, or
- validated — `validation_errors[]`; no representation exists.
- - **No verdict (non-2xx):** a selection refusal — an unknown pipe ref, or an omitted `pipe_ref` that
- nothing defaults (no fetched-manifest `main_pipe`, and a closure declaring no — or several —
- `main_pipe`) — is an input 422 whose `error_type` is `EntryPipeNotFoundError` or
- `EntryPipeAmbiguousError`; a pipe whose `native.Anything` output has no determinable shape, or a
- malformed closure selector, is a request-shape 422 problem+json; a registry-form `method_ref` is a 501 until server-side
- method-registry resolution exists.
-
- An omitted `pipe_ref` defaults the way a run by address does: to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's own declared `main_pipe`.
- """
- try:
- resolved = resolve_requested_crate(request_data)
- except ValidateBundleError as validate_error:
- return invalid_crate_report_response(validate_error.to_error_report())
- try:
- requested_pipe = resolve_requested_pipe(
- resolved.crate,
- pipe_ref=request_data.pipe_ref,
- manifest_main_pipe=resolved.manifest_main_pipe,
- )
- report = _render_report(requested=request_data, requested_pipe=requested_pipe)
- # exclude_none drops the representation field the `format` did not select (and an absent
- # `requested_pipe_ref`), so exactly the fields the caller's own request implies are present.
- return JSONResponse(content=report.model_dump(mode="json", by_alias=True, exclude_none=True))
- finally:
- teardown_current_library()
diff --git a/api/pipelex_api/routes/pipelex/build/runner.py b/api/pipelex_api/routes/pipelex/build/runner.py
deleted file mode 100644
index 03a6bd74d..000000000
--- a/api/pipelex_api/routes/pipelex/build/runner.py
+++ /dev/null
@@ -1,236 +0,0 @@
-from typing import Annotated, Literal, Union
-
-from fastapi import APIRouter, Request
-from fastapi.responses import JSONResponse
-from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION
-from pipelex.base_exceptions import PipelexUnexpectedError
-from pipelex.builder.runner_code import generate_runner_code
-from pipelex.codegen.emission import build_stamped_projection
-from pipelex.codegen.emitters.naming import runtime_to_emitted_class_names
-from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget
-from pipelex.codegen.emitters.types_emitter import emit_types
-from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME
-from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate
-from pipelex.core.pipes.variable_multiplicity import parse_concept_with_multiplicity
-from pipelex.interpreter_hub import get_current_library_id_or_none, get_library_manager
-from pipelex.libraries.crate_normalization import normalize_crate
-from pipelex.libraries.pipe.exceptions import EntryPipeNotFoundError, PipeNotFoundError
-from pipelex.mthds_parsing.pipelex_bundle_blueprint import PipelexBundleBlueprint
-from pipelex.pipeline.bundle_validator import DryRunOutput, DryRunStatus
-from pipelex.pipeline.exceptions import ValidateBundleError
-from pipelex.pipeline.validate_bundle import validate_bundle
-from pipelex.system.caller_identity import CallerIdentity
-from pipelex.tools.misc.package_utils import get_package_version
-from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
-from pydantic import BaseModel, Field
-
-from pipelex_api.errors import raise_validation_error
-from pipelex_api.json_body import JsonBodyRoute
-from pipelex_api.openapi_responses import PROBLEM_404_METHOD_PACKAGE, PROBLEM_501_METHOD_REF
-from pipelex_api.routes.pipelex.crate_ops import (
- CrateInvalidReport,
- GeneratedArtifact,
- invalid_crate_report_response,
- resolve_requested_pipe,
- selected_files,
- teardown_current_library,
-)
-from pipelex_api.routes.pipelex.pipeline import get_request_user_id
-from pipelex_api.schemas.models import ALLOW_SIGNATURES_DESCRIPTION, CallerAnalyticsGroupsMixin, MthdsPipeRequest
-
-router = APIRouter(tags=["build"], route_class=JsonBodyRoute)
-
-
-class BuildRunnerRequest(MthdsPipeRequest, CallerAnalyticsGroupsMixin):
- """The runner-script request: the shared closure + pipe selectors, plus the sweep's `allow_signatures`.
-
- Alone among the `/build/*` projections this route keeps `allow_signatures`, because alone among
- them it still runs the dry-run sweep — the flag only ever parameterized that sweep.
- """
-
- allow_signatures: bool = Field(default=False, description=ALLOW_SIGNATURES_DESCRIPTION)
-
-
-class RunnerStructures(BaseModel):
- """The typed-structures projection the runner script imports from (`from structures.structures import ...`).
-
- The same stamped `python-structures` artifacts + `codegen.lock` a local `pipelex build runner`
- scaffolds into `<output>/structures/` — write `artifacts` and the lock under `directory` and
- the returned `python_code` runs against them, with the offline `codegen check` passing there.
- """
-
- directory: str = Field(default="structures", description="Directory (relative to the runner script) to write the artifacts and lock into.")
- artifacts: list[GeneratedArtifact] = Field(
- default_factory=empty_list_factory_of(GeneratedArtifact),
- description="The stamped generated files (paths relative to `directory`).",
- )
- lock: str = Field(..., description="The lock content (TOML) tracking the artifact set — write verbatim inside `directory`.")
- lock_filename: str = Field(default=CODEGEN_LOCK_FILENAME, description="Filename the lock content must be written as.")
-
-
-class BuildRunnerValidReport(BaseModel):
- """The 200 **valid** arm: the runner script plus the structures projection it imports from."""
-
- is_valid: Literal[True] = True
- pipe_ref: str = Field(..., description="The qualified pipe the runner was generated for — the resolved selector.")
- requested_pipe_ref: str | None = Field(
- default=None,
- description=(
- "The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's "
- "`main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`."
- ),
- )
- python_code: str = Field(..., description="Generated Python script for running the pipeline, imports spelled with the emitted class names.")
- structures: RunnerStructures = Field(..., description="The typed-structures projection the script imports from.")
- message: str = Field(default="Runner code generated successfully", description="Status message")
-
-
-# Discriminated 200 response union: the `/validate` discipline — the verdict rides `is_valid`,
-# never the HTTP status (breaking change from the previous `success`-bool body + 422-on-invalid).
-BuildRunnerResponse = Annotated[Union[BuildRunnerValidReport, CrateInvalidReport], Field(discriminator="is_valid")]
-
-
-def _reject_if_requested_pipe_skipped(sweep_result: dict[str, DryRunOutput], *, pipe_ref: str) -> None:
- """Reject when the *requested* pipe was SKIPPED (its cross-package dependency is unresolved).
-
- `validate_pipes` records a pipe SKIPPED — rather than failing the whole sweep — when a controller
- references a sub-pipe in a package not included in the request (the cross-package tolerance shared
- with `/validate`). For a code-generation endpoint that is a footgun: `generate_runner_code` reads
- only the pipe's own inputs/output and never resolves sub-pipes, so it would happily emit runner
- code for a pipeline that cannot actually run. A SKIPPED requested pipe is a no-verdict condition
- (its dependency closure is absent from the request, so nothing could be diagnosed) → request-shape
- 422; unrelated SKIPPED pipes elsewhere in the bundle stay tolerated.
- """
- for output in sweep_result.values():
- if pipe_ref != output.pipe_ref:
- continue
- match output.status:
- case DryRunStatus.SKIPPED:
- detail = output.error_message or "its dependencies could not be resolved"
- raise_validation_error(f"Cannot generate runner code for pipe '{pipe_ref}': {detail}")
- case DryRunStatus.SUCCESS | DryRunStatus.FAILURE:
- return
-
-
-def _output_is_list(blueprints: list[PipelexBundleBlueprint], *, pipe_ref: str) -> bool:
- """Whether the requested pipe's declared output carries a list multiplicity marker (mirrors the CLI).
-
- Matched on the **qualified** ref. A blueprint's `pipe` map is keyed by bare code, so scanning by
- bare code alone would return the first blueprint declaring that code — which in a multi-domain
- closure can be a *different* pipe that merely shares the name, and whose multiplicity may differ.
- The runner would then be generated with the wrong output handling. Compare the owning domain too.
- """
- domain_code, _, pipe_code = pipe_ref.rpartition(".")
- for blueprint in blueprints:
- if blueprint.domain != domain_code or not blueprint.pipe or pipe_code not in blueprint.pipe:
- continue
- output_parse = parse_concept_with_multiplicity(blueprint.pipe[pipe_code].output)
- return output_parse.multiplicity is not None
- return False
-
-
- "/build/runner",
- response_model=BuildRunnerResponse,
- # On top of the composite router's shared 401/413/422/500: the `method_ref` closure selector the
- # envelope accepts but no server-side method registry resolves yet (shared with /resolve, /codegen).
- responses={404: PROBLEM_404_METHOD_PACKAGE, 501: PROBLEM_501_METHOD_REF},
-)
-async def build_runner(request: Request, request_data: BuildRunnerRequest) -> JSONResponse:
- """Generate a Python runner script for a pipe, riding the codegen types projection.
-
- The one `/build/*` projection that is **not** static: a runner script is a promise the pipe can
- actually run, so this route keeps the dry-run sweep its siblings dropped (and with it
- `allow_signatures`, which only ever parameterized that sweep). `validate_bundle` opens a single
- library, loads the closure, sweeps, and on success leaves the library loaded + current; on failure
- it tears it down itself. On the success path the crate is read from that library, the
- `python-structures` projection is emitted and stamped, and the runner script is generated with the
- **emitted** class names — the same flow as a local `pipelex build runner`.
-
- The sweep is scoped to the requested pipe, so unrelated broken siblings do not block a good pipe.
- When `pipe_ref` is omitted the scope is not settled before the closure loads, so the whole closure
- is swept and the pipe then defaults the way a run by address does — to the fetched package
- manifest's `main_pipe` on a `method_ref` request, else to the closure's own declared `main_pipe` —
- a stricter verdict, and the honest one for a caller who did not say which pipe they meant.
-
- Response contract (the `/validate` discipline): an invalid closure — including a failed dry-run of
- the requested pipe — is a produced verdict: a **200** `is_valid: false` with the structured
- `validation_errors[]`. Non-2xx is reserved for no-verdict conditions: a selection refusal (an
- unknown pipe ref, an omitted `pipe_ref` that nothing defaults — no fetched-manifest `main_pipe`,
- and a closure declaring no, or several, `main_pipe`), an input 422 whose `error_type` is
- `EntryPipeNotFoundError` or `EntryPipeAmbiguousError`; a request-shape 422 for a requested pipe
- whose cross-package dependencies are absent from the request; a 501 for a registry-form `method_ref`, auth, server
- fault — RFC 7807 via the global handlers.
- """
- library_manager = get_library_manager()
- selection = selected_files(request_data)
-
- try:
- # If this raises, validate_bundle has already torn down its own library — nothing to clean up here.
- validate_result = await validate_bundle(
- mthds_contents=[item.content for item in selection.files],
- mthds_sources=[item.source for item in selection.files],
- allow_signatures=request_data.allow_signatures,
- dry_run_pipe_codes=[request_data.pipe_ref] if request_data.pipe_ref else None,
- # The sweep is done for the caller, so its dry runs and `pipe_dry_run` event name them.
- caller_identity=CallerIdentity.make_from_host(user_id=get_request_user_id(request), extras=request_data.analytics_groups),
- )
- except ValidateBundleError as validate_error:
- return invalid_crate_report_response(validate_error.to_error_report())
- except PipeNotFoundError as exc:
- # The engine deliberately lets this one through untranslated (see `translate_to_validate_bundle_error`)
- # so the caller can own it: a pipe ref naming nothing in the closure is an input 422, not an
- # invalid-closure verdict — nothing about the closure is wrong. It is re-raised as the entry-lookup
- # class the slice miss already is, so its `error_type` matches `resolve_requested_pipe`'s refusals.
- msg = f"Pipe '{request_data.pipe_ref}' not found in the submitted closure: {exc}"
- raise EntryPipeNotFoundError(msg) from exc
-
- # Success: validate_bundle left its library loaded + current. Build everything from it, then own its teardown.
- library_id = get_current_library_id_or_none()
- try:
- crate = library_manager.get_crate(library_id) if library_id else None
- if crate is None:
- # Unreachable after a successful in-memory validate (the blueprints were accumulated),
- # so a None crate is an internal invariant break — a server fault (5xx), never a
- # caller-facing verdict (mirrors resolve_crate_from_contents's identical guard).
- msg = "library crate unavailable after a successful bundle load"
- raise PipelexUnexpectedError(msg)
- normalized_crate = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)
- requested_pipe = resolve_requested_pipe(
- normalized_crate,
- pipe_ref=request_data.pipe_ref,
- manifest_main_pipe=selection.manifest_main_pipe,
- )
-
- # The sweep tolerates a cross-package unresolved dependency by recording the pipe SKIPPED
- # instead of failing. Don't hand back runner code for the *requested* pipe in that state.
- _reject_if_requested_pipe_skipped(validate_result.dry_run_result, pipe_ref=requested_pipe.ref)
-
- emitted = emit_types(normalized_crate, target=CodegenTarget.PYTHON_STRUCTURES)
- projection = build_stamped_projection(
- emitted,
- crate_fingerprint=normalized_crate.fingerprint,
- engine_version=get_package_version(),
- kind=CodegenKind.TYPES,
- target=CodegenTarget.PYTHON_STRUCTURES,
- )
- class_name_overrides = runtime_to_emitted_class_names(resolve_concepts_from_crate(normalized_crate))
-
- python_code = generate_runner_code(
- pipe=requested_pipe.pipe,
- output_multiplicity=_output_is_list(validate_result.blueprints, pipe_ref=requested_pipe.ref),
- class_name_overrides=class_name_overrides,
- )
-
- report = BuildRunnerValidReport(
- pipe_ref=requested_pipe.ref,
- requested_pipe_ref=request_data.pipe_ref,
- python_code=python_code,
- structures=RunnerStructures(
- artifacts=[GeneratedArtifact(path=stamped.filename, content=stamped.content) for stamped in projection.files],
- lock=projection.lock_content,
- ),
- )
- return JSONResponse(content=report.model_dump(mode="json", by_alias=True, exclude_none=True))
- finally:
- teardown_current_library()
diff --git a/api/pipelex_api/routes/pipelex/codegen.py b/api/pipelex_api/routes/pipelex/codegen.py
index c52bc0cd8..129f19962 100644
--- a/api/pipelex_api/routes/pipelex/codegen.py
+++ b/api/pipelex_api/routes/pipelex/codegen.py
@@ -32,8 +32,8 @@ class CodegenRouteKind(StrEnum):
Membership follows the **trust chain**: a kind is served here exactly when its artifacts are
stamped and locked, so a client can write them verbatim and pass the offline `codegen check` —
the promise this route's valid arm makes by carrying a `lock`. Input templates are user-editable
- scaffolds, never stamped or locked, so they cannot make that promise; they ride
- `POST /build/inputs` instead and `inputs` is therefore not a kind here. Future per-pipe kinds
+ scaffolds, never stamped or locked, so they cannot make that promise: a client projects one from
+ the input form `POST /pipe-io` serves, and `inputs` is therefore not a kind here. Future per-pipe kinds
(`docs`, `tools`, `tests`) do emit tracked artifacts, so they join this enum and select their
pipe via `pipe_ref`. An unknown kind is a request-shape 422 listing the served set.
"""
diff --git a/api/pipelex_api/routes/pipelex/crate_ops.py b/api/pipelex_api/routes/pipelex/crate_ops.py
index 4062f3874..456fe5896 100644
--- a/api/pipelex_api/routes/pipelex/crate_ops.py
+++ b/api/pipelex_api/routes/pipelex/crate_ops.py
@@ -1,4 +1,4 @@
-"""Shared crate-resolution plumbing for the `/resolve`, `/codegen`, `/pipe-io` and `/build/*` routes.
+"""Shared crate-resolution plumbing for the `/resolve`, `/codegen` and `/pipe-io` routes.
They all select a closure the same way (inline `files[]` XOR `method_ref`), resolve it through the
same engine core (`pipelex.pipeline.resolve_bundle.resolve_crate_from_contents`), and speak the same
@@ -8,10 +8,8 @@ This module holds the pieces they share so the envelopes cannot drift.
The invalid arm here is the *crate* verdict: it deliberately omits `/validate`'s runnability facts
(`pending_signatures`, `is_runnable`) — resolution is static (no dry-run sweep, matching
-`pipelex resolve`), so runnability is not part of its vocabulary. The per-pipe projections
-(`/build/{inputs,output}`) ride that same static core: a template is a read of the pipe's *declared*
-IO, so a valid verdict there says the closure is structurally sound, never that the pipe runs.
-`/build/runner` is the exception — it needs the dry-run sweep, so it keeps `validate_bundle`.
+`pipelex resolve`), so runnability is not part of its vocabulary. The per-pipe route (`/pipe-io`)
+rides that same static core: it reads the selected pipe's *declared* IO with no dry-run sweep.
"""
from enum import StrEnum
@@ -87,15 +85,14 @@ class SelectedFiles(NamedTuple):
def selected_files(request_data: MthdsFilesRequest) -> SelectedFiles:
"""The files the closure selector names: inline `files[]`, or an address `method_ref`'s package.
- Shared by every route on the `files[]` envelope — including `/build/runner`, which cannot use
- `resolve_requested_crate` (it needs `validate_bundle`'s dry-run sweep) but owes the caller the
- same answer on every selector.
+ Every route on the `files[]` envelope reaches it through `resolve_requested_crate`, so they all
+ give the same answer on every selector.
An **address-form** `method_ref` (`github.com/<owner>/<repo>[/<selector>][@<tag>]`) resolves
through the same fetch path the run routes use: the package's `.mthds` files come back as
`files[]` items with their real relative paths as per-file sources, and the manifest's
- `main_pipe` rides beside them so the per-pipe projections can default their selector the way
- a run by address does. Only `.mthds` data travels — the package's Python (if any) never loads
+ `main_pipe` rides beside them so the per-pipe route can default its selector the way a run
+ by address does. Only `.mthds` data travels — the package's Python (if any) never loads
on these routes. The **registry form** (any non-address reference) stays reserved and keeps
its 501 until the packaging program's registry phase.
@@ -159,10 +156,10 @@ def resolve_requested_crate(request_data: MthdsFilesRequest) -> ResolvedClosure:
class RequestedPipe(NamedTuple):
- """The pipe a per-pipe projection was asked for: its resolved qualified ref and the live pipe."""
+ """The pipe a per-pipe route was asked for: its resolved qualified ref and the live pipe."""
ref: str
- """The qualified `domain.pipe_code` actually projected — always qualified, whatever the request spelled."""
+ """The qualified `domain.pipe_code` actually selected — always qualified, whatever the request spelled."""
pipe: PipeAbstract
"""The live pipe, read from the library `resolve_requested_crate` left loaded + current."""
@@ -197,7 +194,7 @@ class _SelectorOrigin(StrEnum):
def resolve_requested_pipe(crate: LibraryCrate, *, pipe_ref: str | None, manifest_main_pipe: str | None) -> RequestedPipe:
- """Select the pipe a per-pipe projection targets, defaulting to the manifest's, then the closure's, `main_pipe`.
+ """Select the pipe a per-pipe route targets, defaulting to the manifest's, then the closure's, `main_pipe`.
The request's `pipe_ref` wins; omitted, the default chain (`select_default_pipe`) decides. Every
failed selection — an unknown ref, an ambiguous one, a chain that finds no entry pipe or several —
diff --git a/api/pipelex_api/routes/pipelex/validate.py b/api/pipelex_api/routes/pipelex/validate.py
index 6cdd388fe..7666f204d 100644
--- a/api/pipelex_api/routes/pipelex/validate.py
+++ b/api/pipelex_api/routes/pipelex/validate.py
@@ -166,13 +166,13 @@ def _effective_default_pipe_ref(report: PipelexValidationReport, *, manifest_mai
The run routes' precedence, minus the request selector `/validate` does not have
(`pipeline.py`: `pipe_code or fetched.main_pipe`, then the batch's primary blueprint via
`select_primary_blueprint`): a fetched package's manifest `main_pipe` wins, and only when there
- is none does the closure's own declaration decide. The per-pipe tooling routes apply that same
+ is none does the closure's own declaration decide. The per-pipe tooling route applies that same
chain in `crate_ops.resolve_requested_pipe`, so a caller reading this field and a caller omitting
- `pipe_ref` on `/build/*` are told about the same pipe wherever `/build/*` defaults at all — see
+ `pipe_ref` on `/pipe-io` are told about the same pipe wherever `/pipe-io` defaults at all — see
the one case below where it refuses to.
- It deliberately states the RUN default, not the build routes' stricter one: a closure whose
- domains each declare a `main_pipe` is refused by `/build/*` but runs happily — `execute` and
+ It deliberately states the RUN default, not the per-pipe route's stricter one: a closure whose
+ domains each declare a `main_pipe` is refused by `/pipe-io` but runs happily — `execute` and
`start` take the first declaring blueprint — so reporting `null` for it would make a consumer
refuse to prepare a method the server would run. The report's `bundle_blueprint` IS that first
declaring blueprint (`build_validation_report` selects it through the same
@@ -374,10 +374,11 @@ async def validate_mthds(request: Request, request_data: ValidateRequest) -> JSO
# so it is never rendered as `is_valid: false` (that verdict is about MTHDS content).
with fetched_method_source(request_data.method_ref) as fetched:
validated_contents = fetched.mthds_contents
- # The manifest's declared entry pipe outranks the closure's own in the run/build default
- # chain, so it must reach `default_pipe_ref` — the report itself is manifest-blind — and
- # the graph arm, which would otherwise draw the primary blueprint's `main_pipe`: a
- # different pipe when the two disagree, and no graph at all when the bundles declare none.
+ # The manifest's declared entry pipe outranks the closure's own in the default chain the
+ # run routes and `/pipe-io` share, so it must reach `default_pipe_ref` — the report itself
+ # is manifest-blind — and the graph arm, which would otherwise draw the primary blueprint's
+ # `main_pipe`: a different pipe when the two disagree, and no graph at all when the bundles
+ # declare none.
# The bare code is passed as is, so the graph arm resolves it exactly as the run does.
manifest_main_pipe = fetched.main_pipe
verdict = await ApiRunner(
diff --git a/api/pipelex_api/schemas/models.py b/api/pipelex_api/schemas/models.py
index bfb94a1ef..c92ba9083 100644
--- a/api/pipelex_api/schemas/models.py
+++ b/api/pipelex_api/schemas/models.py
@@ -283,7 +283,7 @@ _CALLER_ANALYTICS_GROUPS_DESCRIPTION = (
class CallerAnalyticsGroupsMixin(BaseModel):
"""The optional `analytics_groups` of a request whose work is done for a caller without being a run.
- `/validate` and `/build/runner` dry-run the submitted pipes, and the runtime attributes that
+ `/validate` dry-runs the submitted pipes, and the runtime attributes that
telemetry to the caller it is handed (`pipelex.system.caller_identity.CallerIdentity`). The user
comes from the trusted auth layer, as on a run; the groups come from the body, as on a run, and are
refused at the wire with the runtime's own rules.
@@ -569,15 +569,17 @@ class MthdsFilesRequest(BaseModel):
class MthdsPipeRequest(MthdsFilesRequest):
- """Shared base for the per-pipe `/build/*` projections: the closure selector plus a pipe selector.
+ """The base of a per-pipe route's request (`/pipe-io`): the closure selector plus a pipe selector.
The pipe selector is the **qualified** ref `domain.pipe_code`, mirroring `pipelex codegen inputs
--pipe`. It is optional, and an omitted selector defaults with the run routes' precedence: on a
`method_ref` request, to the fetched package manifest's `main_pipe` (the package author's declared
entry pipe); otherwise to the closure's declared `main_pipe`, which is what a single-bundle caller
almost always wants. A closure that declares no `main_pipe` — or several, across domains — cannot
- be defaulted from its own declarations, so an omitted selector is a request-shape 422 there when
- no manifest `main_pipe` settles it (the same two arms the CLI rejects on).
+ be defaulted from its own declarations, so when no manifest `main_pipe` settles it a route that
+ needs one pipe refuses the omitted selector with an input 422, `EntryPipeNotFoundError` or
+ `EntryPipeAmbiguousError` (`crate_ops.resolve_requested_pipe`), never the request-shape
+ `ValidationError` of a malformed body.
"""
pipe_ref: str | None = Field(
@@ -585,9 +587,9 @@ class MthdsPipeRequest(MthdsFilesRequest):
min_length=1,
max_length=MAX_PIPE_CODE_LEN,
description=(
- "Qualified pipe ref (`domain.pipe_code`) to project. Optional — defaults to the fetched package manifest's "
+ "Qualified pipe ref (`domain.pipe_code`) to select. Optional — defaults to the fetched package manifest's "
"`main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`; a closure declaring none, "
- "or several, with no manifest `main_pipe` to settle it, requires it explicitly."
+ "or several, with no manifest `main_pipe` to settle it, requires it unless `all_pipes` is set."
),
)
@@ -596,8 +598,8 @@ ALLOW_SIGNATURES_DESCRIPTION = (
"When true, the validation sweep tolerates unimplemented pipe signatures instead of rejecting the "
"bundle (signatures dry-run trivially by minting a mock). Defaults to false (strict)."
)
-"""Shared by the two routes that still run the dry-run sweep (`/validate`, `/build/runner`) — the flag only
-parameterizes that sweep, so the static `/build/{inputs,output}` projections do not accept it."""
+"""Used by `/validate`, the one route that runs the dry-run sweep: the flag only parameterizes that sweep,
+so the static crate routes do not accept it."""
class MthdsContentsRequest(BaseModel):
@@ -606,7 +608,7 @@ class MthdsContentsRequest(BaseModel):
This is the flat-list envelope. `/validate` uses it through this class (its `ValidateRequest`
subclass adds a parallel `mthds_sources` for per-file source labels), and the run routes
`/execute` and `/start` carry the same `mthds_contents` field independently on `RunRequest`.
- The newer Pipelex-API extension routes (`/resolve`, `/codegen`, `/build/*`) use
+ The newer Pipelex-API extension routes (`/resolve`, `/codegen`, `/pipe-io`) use
`MthdsFilesRequest` instead (`files[]` pairing each content with its `source`, XOR a
`method_ref`), which folds the source label into each entry. `/validate` keeps the flat-list
shape deliberately: it is an MTHDS Protocol route, so changing its inline envelope is a
diff --git a/api/scripts/export_openapi.py b/api/scripts/export_openapi.py
index 7627c769e..95a2a1086 100644
--- a/api/scripts/export_openapi.py
+++ b/api/scripts/export_openapi.py
@@ -3,8 +3,8 @@
The committed file (`docs/api-server/openapi/pipelex-api.openapi.yaml` at the repository root,
published on the pipelex docs site beside the server's pages) is the layer-2 contract of the MTHDS
Protocol nesting (MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex hosted API): the five protocol routes are
-tagged `x-mthds-protocol: true`, the build tooling extensions ride alongside, and the non-contract
-storage routes (`/upload`, `/resolve-storage-url`) are documented as such in their descriptions.
+tagged `x-mthds-protocol: true`, and the untagged Pipelex extensions (`/resolve`, `/codegen`,
+`/pipe-io`, `/lint`, `/format`) ride alongside.
Usage:
python scripts/export_openapi.py ../docs/api-server/openapi/pipelex-api.openapi.yaml
diff --git a/api/tests/unit/_constants.py b/api/tests/unit/_constants.py
index dad43a078..1afb94668 100644
--- a/api/tests/unit/_constants.py
+++ b/api/tests/unit/_constants.py
@@ -139,8 +139,8 @@ Digest this brief: $brief
"""
# A valid single-pipe bundle that declares NO main_pipe — validates fine (D2: no main-pipe
-# precondition on /validate) and simply yields no graph. On the per-pipe `/build/*` projections it is
-# also the closure that cannot default its pipe selector: an omitted `pipe_ref` is a 422 there.
+# precondition on /validate) and simply yields no graph. On `/pipe-io` it is also the closure that
+# cannot default its pipe selector: an omitted `pipe_ref` is a 422 there.
NO_MAIN_PIPE_MTHDS = """\
domain = "nomain"
@@ -166,23 +166,10 @@ output = "Text"
prompt = "@text"
"""
-# A pipe that declares no inputs at all. Its inputs template is empty — a valid verdict, not an error
-# (the engine renderers raise NoInputsRequiredError; the CLI exits 0 on it).
-NO_INPUTS_MTHDS = """\
-domain = "noinputs"
-main_pipe = "greet"
-
-[pipe.greet]
-type = "PipeLLM"
-description = "Greet nobody in particular"
-output = "Text"
-prompt = "Say hello"
-"""
-
# A pipe whose BARE code collides with VALID_MTHDS's `echo`, in a different domain, and whose output
# carries the opposite multiplicity (a list). Submitted alongside VALID_MTHDS it catches any lookup
-# that matches a pipe by bare code alone: `/build/runner` reads the requested pipe's output
-# multiplicity out of the blueprints, and a bare-code scan would return whichever `echo` came first.
+# that matches a pipe by bare code alone: a bare `echo` must be refused as ambiguous, never resolved
+# to whichever `echo` came first.
COLLIDING_ECHO_LIST_MTHDS = """\
domain = "twin"
diff --git a/api/tests/unit/test_allow_signatures.py b/api/tests/unit/test_allow_signatures.py
index 343bbe710..d8a7f5432 100644
--- a/api/tests/unit/test_allow_signatures.py
+++ b/api/tests/unit/test_allow_signatures.py
@@ -3,18 +3,15 @@
After the signatures-as-data reframe, an unimplemented `PipeSignature` is **never** a validation
error — it is a *runnability fact*. `allow_signatures` no longer gates acceptance; it only controls
whether signature pipes are mock-run during the dry-run sweep (and therefore listed in
-`validated_pipes`). So a bundle containing a signature is accepted in BOTH modes on the two routes
-that still run that sweep — `/validate` and `/build/runner`. On `/validate` the outstanding signature
-surfaces as `pending_signatures` + `is_runnable: false` (a 200 `ValidReport`, since the verdict is
-"sound but not runnable"), and the only mode difference is whether the signature pipe appears in
-`validated_pipes`.
+`validated_pipes`). So a bundle containing a signature is accepted in BOTH modes on `/validate`, the
+one route that runs that sweep. The outstanding signature surfaces as `pending_signatures` +
+`is_runnable: false` (a 200 `ValidReport`, since the verdict is "sound but not runnable"), and the
+only mode difference is whether the signature pipe appears in `validated_pipes`.
-The static projections (`/build/{inputs,output}`) dropped the sweep entirely, so they no longer take
-the flag at all — a signature bundle is simply accepted there, and that is asserted in
-`test_build_routes_envelope.py`, not here.
+The static crate routes run no sweep, so they do not take the flag at all.
(`SignaturesNotAllowedError` — the strict-mode rejection this module used to assert — was removed
-with the de-raise; the build routes no longer reject a signature bundle.)
+with the de-raise.)
"""
import pytest
@@ -34,19 +31,12 @@ def _build_client() -> TestClient:
class TestAllowSignatures:
- @pytest.mark.parametrize(
- ("path", "payload_base"),
- [
- ("/v1/validate", {"mthds_contents": [SIGNATURE_MTHDS]}),
- ("/v1/build/runner", {"files": [{"content": SIGNATURE_MTHDS}], "pipe_ref": "sig_api.caller_seq"}),
- ],
- )
@pytest.mark.parametrize("allow_signatures", [False, True], ids=["strict", "lenient"])
- def test_signature_bundle_accepted_in_both_modes(self, path: str, payload_base: dict[str, object], allow_signatures: bool):
- # An unimplemented signature is never a rejection (D-B): both modes answer 200 on the two routes
- # that still sweep. Strict no longer differs from lenient on acceptance.
+ def test_signature_bundle_accepted_in_both_modes(self, allow_signatures: bool):
+ # An unimplemented signature is never a rejection (D-B): both modes answer 200. Strict no
+ # longer differs from lenient on acceptance.
client = _build_client()
- response = client.post(path, json={**payload_base, "allow_signatures": allow_signatures})
+ response = client.post("/v1/validate", json={"mthds_contents": [SIGNATURE_MTHDS], "allow_signatures": allow_signatures})
assert response.status_code == 200, response.text
@pytest.mark.parametrize("allow_signatures", [False, True], ids=["strict", "lenient"])
@@ -77,18 +67,3 @@ class TestAllowSignatures:
assert strict["pending_signatures"] == lenient["pending_signatures"] == ["sig_api.summary_sig"]
assert strict["is_runnable"] is False
assert lenient["is_runnable"] is False
-
- def test_build_runner_generates_code_for_bundle_with_signatures_when_opted_in(self):
- # The most concrete proof the flag threads through to the runner build: with allow_signatures
- # the dry-run sweep mock-runs the signature, the library stays open, and runner code is
- # generated for the caller pipe.
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": SIGNATURE_MTHDS}], "pipe_ref": "sig_api.caller_seq", "allow_signatures": True},
- )
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is True
- assert body["pipe_ref"] == "sig_api.caller_seq"
- assert body["python_code"]
diff --git a/api/tests/unit/test_build_and_agent_routes.py b/api/tests/unit/test_build_and_agent_routes.py
deleted file mode 100644
index 6a05cccd5..000000000
--- a/api/tests/unit/test_build_and_agent_routes.py
+++ /dev/null
@@ -1,382 +0,0 @@
-"""Smoke + validation tests for /validate, /build/* and /build/{concept,pipe-spec,models}."""
-
-from typing import Any
-
-import pytest
-from fastapi import FastAPI
-from fastapi.testclient import TestClient
-from mthds.protocol.models import ModelCategory as MthdsModelCategory
-from pipelex.builder.operations.models_ops import ModelCategory
-from pipelex.interpreter_hub import get_library_manager
-from pipelex.pipe_run.exceptions import DryRunError
-from pipelex.pipeline.bundle_validator import DryRunOutput, DryRunStatus
-from pytest_mock import MockerFixture
-
-from pipelex_api.exception_handlers import register_exception_handlers
-from pipelex_api.routes import router as api_router
-from tests.unit._constants import VALID_MTHDS
-
-
-def _build_client() -> TestClient:
- app = FastAPI()
- app.include_router(api_router, prefix="/v1")
- register_exception_handlers(app)
- return TestClient(app)
-
-
-class TestBuildAndAgentRoutes:
- def test_validate_rejects_oversized_mthds(self):
- client = _build_client()
- oversized = "a" * (2 * 1024 * 1024) # 2 MiB > 1 MiB cap
- response = client.post(
- "/v1/validate",
- json={"mthds_contents": [oversized]},
- )
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- def test_validate_rejects_too_many_files(self):
- client = _build_client()
- response = client.post(
- "/v1/validate",
- json={"mthds_contents": [VALID_MTHDS] * 32},
- )
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- def test_validate_invalid_mthds_returns_200_invalid_report(self):
- # `/validate` is a diagnostic endpoint: an invalid bundle is a produced verdict, so it rides
- # a 200 `InvalidReport` (discriminated on `is_valid: false`) — NOT a 422 problem document,
- # and NOT the older `{success, mthds_contents, message}` 422 envelope. Invalid TOML reliably
- # triggers a `ValidateBundleError` from the interpreter, which the route converts to the
- # invalid arm. The verdict is carried by `is_valid`/`message`, not the status code.
- client = _build_client()
- response = client.post(
- "/v1/validate",
- json={"mthds_contents": ["this is not valid TOML !!!"]},
- )
- assert response.status_code == 200, response.text
- assert response.headers["content-type"].startswith("application/json")
- body = response.json()
- assert body["is_valid"] is False
- assert body["is_runnable"] is False
- # The pipelex message is preserved (caller-facing under `_authors_caller_facing_message`) so
- # the client gets the actual interpreter complaint, not a generic placeholder.
- assert "TOML" in body["message"]
- # The diagnostics list is non-empty on every invalid arm — the structured-info invariant is
- # total. A raw TOML-syntax error is a parse-level failure the interpreter raises with no
- # categorized error-data, but the shared builder's last-resort residual still emits one
- # `blueprint_validation` item carrying the message (no `source` at this layer). A categorized
- # failure (e.g. an invalid `main_pipe`) yields richer items; that path is pinned in
- # test_validate_errors.py.
- assert isinstance(body["validation_errors"], list)
- assert body["validation_errors"], "an invalid verdict must carry a non-empty validation_errors[]"
- assert body["validation_errors"][0]["category"] == "blueprint_validation"
- assert body["validation_errors"][0]["message"]
- # The valid arm's structural artifacts + the retired `success` extra are absent.
- assert "bundle_blueprint" not in body
- assert "success" not in body
-
- # NOTE: the former `main_pipe` precondition on /validate is deleted (protocol alignment
- # D2) — a bundle without `main_pipe` now answers 200 with `graph_spec=null`. The
- # regression pin for that behavior (both backends) lives in `test_validate_envelope.py`.
-
- def test_build_inputs_rejects_oversized_pipe_ref(self):
- client = _build_client()
- long_ref = "x" * 1024
- response = client.post(
- "/v1/build/inputs",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": long_ref},
- )
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- def test_build_concept_rejects_oversized_spec(self):
- client = _build_client()
- big_spec = {"description": "x" * (512 * 1024)} # 512 KiB > 256 KiB cap
- response = client.post("/v1/build/concept", json={"spec": big_spec})
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- @pytest.mark.parametrize("structure", ["string", {"f": 42}])
- def test_build_concept_refuses_malformed_structure_with_a_typed_error(self, structure: Any):
- # `parse_concept_spec` validates the raw shape of `structure` and raises the
- # typed `ConceptSpecError`, so the route answers a problem document rather
- # than an opaque 500 from a bare AttributeError / TypeError.
- client = _build_client()
- response = client.post(
- "/v1/build/concept",
- json={"spec": {"concept_code": "Thing", "description": "d", "structure": structure}},
- )
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- body = response.json()
- assert body["error_type"] == "ConceptSpecError"
- assert body["error_domain"] == "input"
-
- def test_build_pipe_spec_rejects_unknown_pipe_type(self):
- # `parse_pipe_spec` raises a bare `ValueError` for an unknown pipe_type
- # (documented in its docstring). The route must classify that as a
- # caller-input 422 — not let it escape as an opaque 500 through the
- # global `Exception` fallback. Regression for `TODOS.md` Q4.
- client = _build_client()
- response = client.post(
- "/v1/build/pipe-spec",
- json={"pipe_type": "NotARealPipeType", "spec": {"pipe_code": "x", "description": "d", "output": "Text"}},
- )
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- body = response.json()
- assert body["error_type"] == "ValidationError"
- assert "NotARealPipeType" in body["detail"]
-
- @pytest.mark.parametrize("bad_type", ["not-a-real-category", ""])
- def test_models_rejects_invalid_category(self, bad_type: str):
- # The empty string is an explicitly-supplied (invalid) filter value, not an absent
- # param — it must fail loudly like any other unknown category, not silently return
- # the unfiltered deck.
- client = _build_client()
- response = client.get(f"/v1/models?type={bad_type}")
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "InvalidModelCategory"
-
- def test_models_rejects_repeated_type_param(self):
- # D11: the protocol's `type` filter is a single plain value. The old route accepted
- # repeated `?type=` values (list[str] Query) — that extension is dropped, and FastAPI
- # would otherwise silently keep one of the values, so the rejection is explicit.
- # Generic ValidationError (not InvalidModelCategory): both values are valid
- # categories — what's wrong is the arity.
- client = _build_client()
- response = client.get("/v1/models?type=llm&type=extract")
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- def test_models_returns_protocol_deck(self):
- # The deck is the protocol `ModelDeck` shape produced by `PipelexMTHDSProtocol.models`:
- # a non-empty flat `models` list (regression for the silently-empty deck the old raw
- # per-category payload caused in the SDK — F2) plus this implementation's routing
- # extensions, keyed by category (the same alias name exists in several categories —
- # a flat map would silently drop entries on collision). The old raw keys (`presets`
- # by category, `success`) are gone. Every entry of the flat list is typed by one of the
- # protocol's categories, `judgment` among them, and never by a value the protocol does not
- # define; the extensions are keyed by the categories this runtime serves.
- client = _build_client()
- response = client.get("/v1/models")
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["models"], "the protocol deck must carry a non-empty models list"
- first_model = body["models"][0]
- assert first_model["name"]
- assert first_model["type"]
- assert {model["type"] for model in body["models"]} <= {category.value for category in MthdsModelCategory}
- valid_categories = {category.value for category in ModelCategory}
- assert set(body["aliases"]) <= valid_categories
- assert all(isinstance(category_aliases, dict) for category_aliases in body["aliases"].values())
- assert set(body["waterfalls"]) <= valid_categories
- assert "presets" not in body
- assert "success" not in body
-
- def test_models_single_type_filter(self):
- client = _build_client()
- response = client.get("/v1/models?type=llm")
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["models"]
- assert {model["type"] for model in body["models"]} == {"llm"}
- assert set(body["aliases"]) <= {"llm"}
-
- def test_empty_mthds_contents_rejected(self):
- # `/validate` is the last route on the `mthds_contents` envelope (it is protocol-owned; the
- # build routes moved to `files[]`, whose empty-selector rejection lives in the envelope suite).
- client = _build_client()
- response = client.post("/v1/validate", json={"mthds_contents": []})
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- def test_build_runner_tears_down_library_when_success_path_raises(self, mocker: MockerFixture):
- # The loaded-on-success contract makes the route own teardown of validate_bundle's library.
- # A failure anywhere in the success window (here: the SKIPPED guard, the first statement
- # inside the try) must NOT leak that library — the finally tears down the exact id
- # validate_bundle opened.
- library_manager = get_library_manager()
- open_spy = mocker.spy(library_manager, "open_library")
- teardown_spy = mocker.spy(library_manager, "teardown")
- mocker.patch(
- "pipelex_api.routes.pipelex.build.runner._reject_if_requested_pipe_skipped",
- side_effect=RuntimeError("synthetic success-path failure"),
- )
-
- # raise_server_exceptions=False: the synthetic RuntimeError reaches the
- # catch-all `Exception` handler, but Starlette's ServerErrorMiddleware
- # would otherwise re-raise it through the TestClient in test mode. The
- # convention matches `test_exception_handlers.py`.
- app = FastAPI()
- app.include_router(api_router, prefix="/v1")
- register_exception_handlers(app)
- client = TestClient(app, raise_server_exceptions=False)
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
-
- # The synthetic RuntimeError propagates to the global Exception handler.
- assert response.status_code == 500
- # validate_bundle DID open a library and leave it current — capture its id.
- assert open_spy.spy_return is not None
- created_library_id, _ = open_spy.spy_return
- # The fix: teardown runs anyway, with the exact id open_library returned.
- teardown_spy.assert_called_once_with(library_id=created_library_id)
-
- def test_build_runner_none_crate_is_a_server_fault_not_a_verdict(self, mocker: MockerFixture):
- # The "unreachable" None-crate guard is an internal invariant break — a server fault (5xx),
- # never a caller-facing verdict. Before the fix it raised ValidateBundleError OUTSIDE the
- # except that maps it to the 200 invalid arm, so it rendered as a request-shape-style 422,
- # contradicting the route's own contract. The library must still be torn down (no leak).
- library_manager = get_library_manager()
- open_spy = mocker.spy(library_manager, "open_library")
- teardown_spy = mocker.spy(library_manager, "teardown")
- mocker.patch.object(library_manager, "get_crate", return_value=None)
-
- app = FastAPI()
- app.include_router(api_router, prefix="/v1")
- register_exception_handlers(app)
- client = TestClient(app, raise_server_exceptions=False)
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
-
- assert response.status_code == 500, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert open_spy.call_count >= 1
- assert open_spy.call_count == teardown_spy.call_count
-
- def test_build_runner_succeeds_and_returns_python_code_and_structures(self):
- # /build/runner rides validate_bundle (loaded-on-success) and the codegen types projection
- # (D9): a 200 valid arm proves the sweep passed, the library survived for code generation,
- # and the response carries BOTH the runner script and the stamped structures projection the
- # script imports from (structures.py + codegen.lock), the retired `success` bool gone.
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is True
- assert "success" not in body
- assert body["pipe_ref"] == "smoke.echo"
- assert body["python_code"]
- assert "echo" in body["python_code"]
- structures = body["structures"]
- assert structures["directory"] == "structures"
- artifact_paths = [artifact["path"] for artifact in structures["artifacts"]]
- assert artifact_paths == ["structures.py"]
- assert structures["artifacts"][0]["content"].startswith("# >>> pipelex-codegen-stamp >>>")
- assert structures["lock_filename"] == "codegen.lock"
- assert "crate_fingerprint" in structures["lock"]
-
- def test_build_runner_keeps_library_open_for_codegen_then_tears_down_once(self, mocker: MockerFixture):
- # The loaded-on-success contract (D6): the inner sweep must NOT tear the library down — if it
- # did, get_required_pipe + generate_runner_code would have failed before the response. Teardown
- # happens exactly once, in the route's finally, after code generation.
- library_manager = get_library_manager()
- teardown_spy = mocker.spy(library_manager, "teardown")
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
- assert response.status_code == 200, response.text
- teardown_spy.assert_called_once()
-
- # NOTE: the "opens exactly one library, tears down that same id" property (C-2 / Q-5) is pinned for
- # all three build routes in `test_build_routes_envelope.py`, which covers both cores they now ride.
-
- def test_build_runner_rejects_when_requested_pipe_is_skipped(self, mocker: MockerFixture):
- # C-3: a cross-package unresolved dependency makes validate_pipes record the requested pipe
- # SKIPPED (not a hard failure), so the sweep returns normally. generate_runner_code reads only
- # the pipe's own inputs/output, so without a guard the route would emit runner code for a
- # pipeline that cannot run. The guard must reject the requested-pipe-SKIPPED case with a 422
- # and never reach code generation. (Other, unrelated SKIPPED pipes stay tolerated.)
- skipped_result = {
- "smoke.echo": DryRunOutput(
- pipe_code="echo",
- pipe_ref="smoke.echo",
- status=DryRunStatus.SKIPPED,
- error_message="Skipped dry run for pipe 'smoke.echo': unresolved dependency: other_pkg.missing",
- )
- }
- # The sweep now runs inside validate_bundle (the route no longer calls it directly), so the
- # SKIPPED outcome is planted at its source module.
- mocker.patch(
- "pipelex.pipeline.validate_bundle.BundleValidator.validate_pipes",
- new=mocker.AsyncMock(return_value=skipped_result),
- )
- generate_spy = mocker.patch("pipelex_api.routes.pipelex.build.runner.generate_runner_code")
-
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
-
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- body = response.json()
- assert body["error_type"] == "ValidationError"
- assert body["error_domain"] == "input"
- assert "echo" in body["detail"]
- # The guard short-circuits before code generation.
- generate_spy.assert_not_called()
-
- def test_build_runner_dry_run_failure_is_a_200_invalid_verdict(self, mocker: MockerFixture):
- # The `/validate` discipline on the build routes: a failed dry-run of a caller-submitted
- # bundle is a *produced negative verdict*, not a transport failure. validate_bundle's shared
- # cascade translates the DryRunError to ValidateBundleError, which the route renders as the
- # 200 invalid arm (is_valid: false + structured validation_errors[]) — the same wire shape
- # /build/inputs and /build/output return for the identical failure. (Breaking change from
- # the previous 422 problem+json.)
- mocker.patch(
- "pipelex.pipeline.validate_bundle.BundleValidator.validate_pipes",
- new=mocker.AsyncMock(side_effect=DryRunError("Dry run failed with 1 unexpected pipe failure(s): 'smoke.echo'")),
- )
- generate_spy = mocker.patch("pipelex_api.routes.pipelex.build.runner.generate_runner_code")
-
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"},
- )
-
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is False
- assert body["validation_errors"], "an invalid verdict must carry a non-empty validation_errors[]"
- assert "python_code" not in body
- # The failure short-circuits before code generation.
- generate_spy.assert_not_called()
-
- def test_validate_tears_down_validate_bundle_library_without_leaking(self, mocker: MockerFixture):
- # validate_bundle opens a library and leaves it loaded + current on success (the D6 contract);
- # /validate must own that teardown. Before the fix the route never tore it down, orphaning it on
- # every successful call (the best-effort graph dry-run opens + tears down its OWN library via
- # PipelexRunner, so that one was balanced). Assert the conservation property: every library opened
- # during the request is torn down — open count == teardown count. A leak shows up as opens > teardowns.
- library_manager = get_library_manager()
- open_spy = mocker.spy(library_manager, "open_library")
- teardown_spy = mocker.spy(library_manager, "teardown")
-
- client = _build_client()
- response = client.post("/v1/validate", json={"mthds_contents": [VALID_MTHDS]})
-
- assert response.status_code == 200, response.text
- assert open_spy.call_count >= 1
- assert open_spy.call_count == teardown_spy.call_count
diff --git a/api/tests/unit/test_build_routes_envelope.py b/api/tests/unit/test_build_routes_envelope.py
deleted file mode 100644
index 9a81384fa..000000000
--- a/api/tests/unit/test_build_routes_envelope.py
+++ /dev/null
@@ -1,427 +0,0 @@
-"""Envelope tests for the migrated `/v1/build/*` projections.
-
-Pins the Phase-2/3 migration: the three build routes ride the shared `files[]` XOR `method_ref`
-closure selector (`MthdsFilesRequest`) plus an optional qualified `pipe_ref` that defaults with the
-run routes' precedence — the fetched package manifest's `main_pipe` on a `method_ref` request, else
-the closure's declared `main_pipe`; `/build/{inputs,output}` resolve their crate **statically** (no
-dry-run sweep, so no `allow_signatures`), while `/build/runner` keeps both. Verdicts stay on the
-`/validate` discipline — 200 discriminated on `is_valid`, non-2xx only for no-verdict conditions.
-"""
-
-from collections.abc import Callable
-from pathlib import Path
-
-import pytest
-from fastapi import FastAPI
-from fastapi.testclient import TestClient
-from pipelex.interpreter_hub import get_library_manager
-from pytest_mock import MockerFixture
-
-from pipelex_api.exception_handlers import register_exception_handlers
-from pipelex_api.routes import router as api_router
-from tests.unit._constants import (
- COLLIDING_ECHO_LIST_MTHDS,
- INVALID_MAIN_PIPE_MTHDS,
- NO_INPUTS_MTHDS,
- NO_MAIN_PIPE_MTHDS,
- SECOND_MAIN_PIPE_MTHDS,
- SIGNATURE_MTHDS,
- STUB_METHOD_ADDRESS,
- STUB_METHOD_MANIFEST_MAIN_PIPE_SHOUT,
- STUB_METHOD_MANIFEST_NO_MAIN_PIPE,
- VALID_MTHDS,
-)
-
-BUILD_PATHS = ["/v1/build/inputs", "/v1/build/output", "/v1/build/runner"]
-
-# The address-form reference the stubbed clone (`install_method_package`) answers to.
-STUB_METHOD_REF = f"{STUB_METHOD_ADDRESS}@v0.1.0"
-
-
-def _build_client() -> TestClient:
- app = FastAPI()
- app.include_router(api_router, prefix="/v1")
- register_exception_handlers(app)
- return TestClient(app)
-
-
-class TestBuildRoutesEnvelope:
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_pipe_ref_defaults_to_the_closures_main_pipe(self, path: str):
- # D1: the selector is optional. Omitted, it resolves to the closure's declared main_pipe, and
- # the valid arm echoes BOTH the resolved ref and (absent) the requested one, so a caller can
- # see what it was defaulted to.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}]})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is True
- assert body["pipe_ref"] == "smoke.echo"
- assert "requested_pipe_ref" not in body, "an omitted selector must not be echoed back as if it were requested"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_explicit_qualified_pipe_ref_is_echoed_as_requested_and_resolved(self, path: str):
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == "smoke.echo"
- assert body["requested_pipe_ref"] == "smoke.echo"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_unknown_pipe_ref_is_a_422_not_an_invalid_verdict(self, path: str):
- # Nothing about the *closure* is wrong — the caller named a pipe that isn't in it. That is a
- # no-verdict condition (422), never a 200 `is_valid: false`.
- # It carries the entry-lookup `error_type` the run routes answer for an unknown `pipe_code`, never
- # a request-shape `ValidationError` — `/build/runner` included, whose sliced sweep reaches it first.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.does_not_exist"})
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- problem = response.json()
- assert problem["error_type"] == "EntryPipeNotFoundError"
- assert problem["error_domain"] == "input"
- assert "not found" in problem["detail"]
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_an_ambiguous_pipe_ref_is_a_422_that_says_ambiguous_not_missing(self, path: str):
- # `echo` is declared by both `smoke` and `twin`, so the engine's entry lookup refuses to pick one.
- # The 422 must say so and name the candidates — calling the pipe "not found" beside a list of
- # the pipes it matched would contradict itself.
- client = _build_client()
- payload = {"files": [{"content": VALID_MTHDS}, {"content": COLLIDING_ECHO_LIST_MTHDS}], "pipe_ref": "echo"}
- response = client.post(path, json=payload)
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "EntryPipeAmbiguousError"
- detail = response.json()["detail"]
- assert "several domains" in detail
- assert "smoke.echo" in detail
- assert "twin.echo" in detail
- assert "not found" not in detail, detail
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_omitted_pipe_ref_on_a_closure_with_no_main_pipe_is_422(self, path: str):
- # First of the two arms `inputs_cmd.py::_default_main_pipe_ref` rejects: nothing to default to.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": NO_MAIN_PIPE_MTHDS}]})
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "EntryPipeNotFoundError"
- assert "main_pipe" in response.json()["detail"]
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_omitted_pipe_ref_on_an_ambiguous_closure_is_422(self, path: str):
- # The second arm — the one the plan's D1 originally missed: two domains, two main_pipes, so the
- # default is ambiguous. Naming the pipe explicitly resolves it (asserted below).
- client = _build_client()
- payload = {"files": [{"content": VALID_MTHDS}, {"content": SECOND_MAIN_PIPE_MTHDS}]}
- response = client.post(path, json=payload)
- assert response.status_code == 422, response.text
- assert response.json()["error_type"] == "EntryPipeAmbiguousError"
- assert "several" in response.json()["detail"]
-
- named = client.post(path, json={**payload, "pipe_ref": "other.shout"})
- assert named.status_code == 200, named.text
- assert named.json()["pipe_ref"] == "other.shout"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_invalid_closure_is_a_200_invalid_verdict(self, path: str):
- client = _build_client()
- response = client.post(path, json={"files": [{"content": INVALID_MAIN_PIPE_MTHDS}]})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is False
- assert body["validation_errors"], "an invalid verdict must carry a non-empty validation_errors[]"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_method_ref_is_501_until_the_registry_exists(self, path: str):
- # The build routes inherit the closure selector whole — including the arm this server does not
- # serve yet. `/build/runner` reaches it through `selected_files` despite not using the crate core.
- client = _build_client()
- response = client.post(path, json={"method_ref": "acme/methods/x"})
- assert response.status_code == 501, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "MethodRefNotSupported"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_method_ref_omitted_pipe_ref_defaults_to_the_manifest_main_pipe(self, path: str, install_method_package: Callable[..., Path]):
- # The fetched package's closure declares NO main_pipe — on its own it cannot default the
- # selector (the 422 pinned above for inline files). But the package manifest declares one
- # (`STUB_METHOD_MANIFEST`: `main_pipe = "echo"`), and that is the package author's entry pipe:
- # the run routes already honor it, so an omitted `pipe_ref` here resolves to it too, and is
- # echoed qualified — a manifest `main_pipe` is a bare code riding the cross-domain fallback.
- install_method_package(files={"documents.mthds": NO_MAIN_PIPE_MTHDS})
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is True
- assert body["pipe_ref"] == "nomain.echo"
- assert "requested_pipe_ref" not in body, "a manifest-defaulted selector must not be echoed back as if it were requested"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_manifest_main_pipe_outranks_the_closures_own_declarations(self, path: str, install_method_package: Callable[..., Path]):
- # Two domains, two declared main_pipes: by the closure's own declarations the default is
- # ambiguous (the 422 pinned above for inline files). The manifest names `shout`, and it sits
- # above the closure in the chain — `pipe_ref or manifest main_pipe or closure main_pipe`, the
- # run routes' `pipe_code or fetched.main_pipe` — so it both settles the ambiguity and wins
- # over `smoke`'s own `echo`.
- install_method_package(
- files={"smoke.mthds": VALID_MTHDS, "other.mthds": SECOND_MAIN_PIPE_MTHDS},
- manifest_toml=STUB_METHOD_MANIFEST_MAIN_PIPE_SHOUT,
- )
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == "other.shout"
- assert "requested_pipe_ref" not in body
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_explicit_pipe_ref_outranks_the_manifest_main_pipe(self, path: str, install_method_package: Callable[..., Path]):
- # The request's own selector stays first in the chain: the manifest says `echo`, the caller
- # says `other.shout`, and the caller wins — echoed as requested and resolved.
- install_method_package(files={"smoke.mthds": VALID_MTHDS, "other.mthds": SECOND_MAIN_PIPE_MTHDS})
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF, "pipe_ref": "other.shout"})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == "other.shout"
- assert body["requested_pipe_ref"] == "other.shout"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_a_manifest_without_main_pipe_falls_back_to_the_closures_declaration(self, path: str, install_method_package: Callable[..., Path]):
- # A manifest may declare no `main_pipe`. Then the chain falls through to the closure's own
- # declaration — the same default inline `files[]` get — rather than refusing.
- install_method_package(files={"documents.mthds": VALID_MTHDS}, manifest_toml=STUB_METHOD_MANIFEST_NO_MAIN_PIPE)
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == "smoke.echo"
- assert "requested_pipe_ref" not in body
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_method_ref_with_nothing_to_default_from_is_still_422(self, path: str, install_method_package: Callable[..., Path]):
- # The whole chain empty: no `pipe_ref`, a manifest with no `main_pipe`, a closure declaring
- # none. The closure-declaration arm's 422 is unchanged — the manifest only ever ADDS a default.
- install_method_package(files={"documents.mthds": NO_MAIN_PIPE_MTHDS}, manifest_toml=STUB_METHOD_MANIFEST_NO_MAIN_PIPE)
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert "main_pipe" in response.json()["detail"]
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_a_manifest_main_pipe_absent_from_the_closure_is_a_422_naming_its_origin(self, path: str, install_method_package: Callable[..., Path]):
- # The manifest names `echo` but the package's closure only contains `other.shout`. Nothing
- # about the closure is wrong, so it is the same request-shape 422 as an unknown explicit ref —
- # but the caller never spelled `echo`, so the detail must say where it came from.
- install_method_package(files={"other.mthds": SECOND_MAIN_PIPE_MTHDS})
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "EntryPipeNotFoundError"
- detail = response.json()["detail"]
- assert "'echo'" in detail
- assert "manifest" in detail, f"the 422 must say the selector came from the manifest: {detail}"
- assert "not found" in detail
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_an_ambiguous_manifest_main_pipe_is_a_422_that_says_ambiguous_not_missing(self, path: str, install_method_package: Callable[..., Path]):
- # The manifest names `echo`, which both `smoke` and `twin` declare. The caller never spelled it,
- # so the detail names the manifest as its origin, and it says the code is ambiguous, not missing.
- install_method_package(files={"smoke.mthds": VALID_MTHDS, "twin.mthds": COLLIDING_ECHO_LIST_MTHDS})
- client = _build_client()
- response = client.post(path, json={"method_ref": STUB_METHOD_REF})
- assert response.status_code == 422, response.text
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "EntryPipeAmbiguousError"
- detail = response.json()["detail"]
- assert "manifest" in detail
- assert "several domains" in detail
- assert "smoke.echo" in detail
- assert "twin.echo" in detail
- assert "not found" not in detail, detail
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_source_labels_ride_through_to_diagnostics(self, path: str):
- # The point of the envelope migration for the build routes: a per-file `source` label reaches the
- # engine, so an invalid closure's diagnostics can name the owning file. (Before Phase 2 the build
- # routes took bare strings and could not carry one — `/build/runner` needed the widened
- # `validate_bundle(mthds_sources=...)` to get here.)
- client = _build_client()
- response = client.post(path, json={"files": [{"content": INVALID_MAIN_PIPE_MTHDS, "source": "bundles/broken.mthds"}]})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is False
- sources = [error.get("source") for error in body["validation_errors"]]
- assert "bundles/broken.mthds" in sources, f"no diagnostic carried the submitted source: {body['validation_errors']}"
-
- @pytest.mark.parametrize("path", ["/v1/build/inputs", "/v1/build/output"])
- def test_static_routes_ignore_allow_signatures(self, path: str):
- # `allow_signatures` only ever parameterized the dry-run sweep. The static projections dropped
- # the sweep, so the flag is gone from their request models — and since the envelope does not
- # forbid extras, a client still sending it is simply ignored rather than rejected.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}], "allow_signatures": True})
- assert response.status_code == 200, response.text
- assert "allow_signatures" not in response.json()
-
- @pytest.mark.parametrize("path", ["/v1/build/inputs", "/v1/build/output"])
- @pytest.mark.parametrize("pipe_ref", ["sig_api.caller_seq", "sig_api.summary_sig"], ids=["caller", "the-signature-itself"])
- def test_static_routes_accept_a_signature_bundle_unconditionally(self, path: str, pipe_ref: str):
- # The corollary of dropping the sweep: an unimplemented `PipeSignature` is a *runnability*
- # fact, and these routes no longer speak runnability. So a signature bundle projects fine with
- # no flag to set — where `/validate` and `/build/runner` still surface it (see
- # test_allow_signatures.py, whose docstring points here for exactly this assertion).
- # Both the caller *and* the signature pipe itself: a signature declares inputs/output, so it is
- # statically projectable precisely because these routes read only the declaration.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": SIGNATURE_MTHDS}], "pipe_ref": pipe_ref})
- assert response.status_code == 200, response.text
- assert response.json()["is_valid"] is True
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_a_bare_pipe_code_still_resolves_but_is_echoed_qualified(self, path: str):
- # The engine's pipe lookup accepts a bare code, falling back across domains. A caller who leans
- # on that must still be told the QUALIFIED ref — the valid arms promise one, so echoing the
- # request's own bare spelling back would quietly break that promise. The resolved ref is read
- # off the live pipe, never off the request.
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "echo"})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == "smoke.echo", "the resolved ref must be qualified, not the bare code the caller sent"
- assert body["requested_pipe_ref"] == "echo", "the echo of the request must stay verbatim"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_empty_files_rejected(self, path: str):
- client = _build_client()
- response = client.post(path, json={"files": []})
- assert response.status_code == 422
- assert response.headers["content-type"] == "application/problem+json"
- assert response.json()["error_type"] == "ValidationError"
-
- @pytest.mark.parametrize("path", BUILD_PATHS)
- def test_routes_open_exactly_one_library_and_tear_it_down(self, path: str, mocker: MockerFixture):
- # The loaded-on-success contract: whichever core a route rides (the static
- # `resolve_requested_crate` for inputs/output, `validate_bundle` for runner), it opens exactly
- # ONE library, leaves it loaded + current, and the route owns its teardown. A second open would
- # orphan the first; a missing teardown would leak it.
- library_manager = get_library_manager()
- open_spy = mocker.spy(library_manager, "open_library")
- teardown_spy = mocker.spy(library_manager, "teardown")
-
- client = _build_client()
- response = client.post(path, json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"})
-
- assert response.status_code == 200, response.text
- assert open_spy.call_count == 1
- created_library_id, _ = open_spy.spy_return
- teardown_spy.assert_called_once_with(library_id=created_library_id)
-
- def test_inputs_json_returns_the_parsed_light_template(self):
- client = _build_client()
- response = client.post("/v1/build/inputs", json={"files": [{"content": VALID_MTHDS}]})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["format"] == "json"
- assert body["explicit"] is False
- # The light shape: a Text-refining input is a bare string, not a {concept, content} envelope.
- assert isinstance(body["inputs"]["text"], str)
- assert "inputs_toml" not in body, "the format the caller did not ask for must be absent"
-
- def test_inputs_explicit_returns_the_ceremonial_envelope(self):
- client = _build_client()
- response = client.post("/v1/build/inputs", json={"files": [{"content": VALID_MTHDS}], "explicit": True})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["explicit"] is True
- assert set(body["inputs"]["text"]) == {"concept", "content"}
-
- @pytest.mark.parametrize("explicit", [False, True], ids=["light", "explicit"])
- def test_inputs_toml_returns_raw_text_not_a_parsed_dict(self, explicit: bool):
- # D3's response-shape consequence: TOML cannot ride as a parsed dict (it would lose the concept
- # comments and key order that are the reason to ask for TOML), so it rides its own string field.
- client = _build_client()
- response = client.post("/v1/build/inputs", json={"files": [{"content": VALID_MTHDS}], "format": "toml", "explicit": explicit})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["format"] == "toml"
- assert isinstance(body["inputs_toml"], str)
- assert "text" in body["inputs_toml"]
- assert "inputs" not in body, "the JSON field must be absent on a TOML request"
-
- @pytest.mark.parametrize("template_format", ["json", "toml"])
- def test_a_pipe_with_no_inputs_is_a_valid_verdict_with_an_empty_template(self, template_format: str):
- # NoInputsRequiredError out of the engine renderers is not a failure — it is the honest answer
- # "run this with nothing". The CLI exits 0 on it; the route answers a valid arm.
- client = _build_client()
- response = client.post("/v1/build/inputs", json={"files": [{"content": NO_INPUTS_MTHDS}], "format": template_format})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["is_valid"] is True
- assert "no inputs" in body["message"]
- empty = body["inputs"] if template_format == "json" else body["inputs_toml"]
- assert not empty
-
- @pytest.mark.parametrize(
- ("output_format", "expected_field"),
- [("schema", "output"), ("json", "output"), ("python", "output_python")],
- )
- def test_output_formats_land_on_the_field_the_format_implies(self, output_format: str, expected_field: str):
- # Regression: `format=python` used to be a hard 500 — the route fed Python source to json.loads
- # and typed the field `dict`. The structured formats stay objects; python is source text.
- client = _build_client()
- response = client.post("/v1/build/output", json={"files": [{"content": VALID_MTHDS}], "format": output_format})
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["format"] == output_format
- assert expected_field in body
- if expected_field == "output":
- assert isinstance(body["output"], dict)
- assert "output_python" not in body
- else:
- assert isinstance(body["output_python"], str)
- assert "output" not in body
-
- @pytest.mark.parametrize(
- ("pipe_ref", "expects_list"),
- [("smoke.echo", False), ("twin.echo", True)],
- ids=["scalar-output", "list-output"],
- )
- def test_runner_reads_output_multiplicity_from_the_right_domain(self, pipe_ref: str, expects_list: bool):
- # Two domains declare a pipe with the same BARE code `echo`, with OPPOSITE output multiplicity.
- # The runner reads the requested pipe's multiplicity out of the blueprints, whose `pipe` maps are
- # keyed by bare code — so a bare-code scan returns whichever blueprint came first and can emit a
- # runner with the wrong output handling. The lookup must compare the owning domain too.
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}, {"content": COLLIDING_ECHO_LIST_MTHDS}], "pipe_ref": pipe_ref},
- )
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["pipe_ref"] == pipe_ref
- # The multiplicity decides the script's result cast: `main_stuff_as_items(item_type=...)` for a
- # list output vs `main_stuff_as(content_type=...)` for a scalar one.
- python_code = body["python_code"]
- assert ("main_stuff_as_items(" in python_code) is expects_list, f"wrong output multiplicity for {pipe_ref}:\n{python_code}"
-
- def test_runner_still_sweeps_and_still_takes_allow_signatures(self, mocker: MockerFixture):
- # `/build/runner` is the one projection that keeps the dry-run (a runner script is a promise the
- # pipe runs), so it alone still accepts the flag that parameterizes that sweep.
- spy = mocker.spy(get_library_manager(), "open_library")
- client = _build_client()
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo", "allow_signatures": True},
- )
- assert response.status_code == 200, response.text
- body = response.json()
- assert body["python_code"]
- assert body["structures"]["artifacts"]
- assert spy.call_count == 1
diff --git a/api/tests/unit/test_codegen_route.py b/api/tests/unit/test_codegen_route.py
index dcf2806c6..f65ba4ac4 100644
--- a/api/tests/unit/test_codegen_route.py
+++ b/api/tests/unit/test_codegen_route.py
@@ -94,7 +94,7 @@ class TestCodegenRoute:
[
{"kind": "does_not_exist"},
{"target": "not-a-target"},
- {"kind": "inputs"}, # served by /build/inputs, deliberately not by this route
+ {"kind": "inputs"}, # an editable scaffold, projected client-side from /pipe-io's input form
{"pipe_ref": "smoke.echo"}, # types is concept-set-wide; a pipe selector is a shape error
],
ids=["unknown-kind", "unknown-target", "unserved-kind", "pipe-ref-on-types"],
diff --git a/api/tests/unit/test_pipe_io_route.py b/api/tests/unit/test_pipe_io_route.py
index 805fdbf2f..ef3e25a7a 100644
--- a/api/tests/unit/test_pipe_io_route.py
+++ b/api/tests/unit/test_pipe_io_route.py
@@ -20,6 +20,7 @@ from pipelex.pipeline.exceptions import PipeIOContractError
from pytest_mock import MockerFixture
from pipelex_api.exception_handlers import register_exception_handlers
+from pipelex_api.limits import MAX_PIPE_CODE_LEN
from pipelex_api.routes import router as api_router
from tests.unit._constants import (
COLLIDING_ECHO_LIST_MTHDS,
@@ -33,6 +34,7 @@ from tests.unit._constants import (
SIGNATURE_ONLY_BATCH,
STUB_METHOD_ADDRESS,
STUB_METHOD_MANIFEST_MAIN_PIPE_SHOUT,
+ STUB_METHOD_MANIFEST_NO_MAIN_PIPE,
STUB_METHOD_MANIFEST_NOMAIN_ENTRY,
VALID_MTHDS,
)
@@ -368,6 +370,44 @@ class TestPipeIoRoute:
client = _build_client()
_assert_input_422(client.post(PIPE_IO_PATH, json=payload), error_type=REQUEST_SHAPE_ERROR)
+ @pytest.mark.parametrize(
+ "payload",
+ [
+ pytest.param({"files": []}, id="empty-files"),
+ pytest.param({"files": [{"content": VALID_MTHDS}], "pipe_ref": "x" * (MAX_PIPE_CODE_LEN + 1)}, id="oversized-pipe-ref"),
+ ],
+ )
+ def test_out_of_bounds_selectors_are_a_request_shape_422(self, payload: dict[str, Any]):
+ client = _build_client()
+ _assert_input_422(client.post(PIPE_IO_PATH, json=payload), error_type=REQUEST_SHAPE_ERROR)
+
+ def test_a_bare_pipe_code_resolves_but_is_answered_qualified(self):
+ # The engine's entry lookup still accepts a bare request code, falling back across domains, until
+ # `resolve_requested_pipe` refuses one as the pipe-selector ruling asks; that change flips this
+ # test. Meanwhile the valid arm promises a qualified ref, so the answer is read off the resolved
+ # pipe, never echoed from the request's own bare spelling.
+ client = _build_client()
+ body = _valid_arm(client, {"files": _files(VALID_MTHDS, SIBLING_MTHDS), "pipe_ref": "wrap_echo"})
+ assert body["pipe_ref"] == "smoke.wrap_echo"
+ _assert_keys(body, {"smoke.wrap_echo"})
+
+ def test_a_manifest_without_main_pipe_falls_through_to_the_closures_declaration(self, install_method_package: Callable[..., Path]):
+ # A manifest may declare no `main_pipe`; the chain then falls through to the closure's own
+ # declaration, the default inline `files[]` get, rather than refusing.
+ install_method_package(files={"documents.mthds": VALID_MTHDS}, manifest_toml=STUB_METHOD_MANIFEST_NO_MAIN_PIPE)
+ client = _build_client()
+ body = _valid_arm(client, {"method_ref": STUB_METHOD_REF})
+ assert body["pipe_ref"] == "smoke.echo"
+ assert body["default_pipe_ref"] == "smoke.echo"
+
+ def test_a_method_with_no_entry_pipe_anywhere_is_still_a_422(self, install_method_package: Callable[..., Path]):
+ # The whole chain empty: no `pipe_ref`, a manifest with no `main_pipe`, a closure declaring
+ # none. The manifest only ever adds a default, so the closure arm's refusal stands.
+ install_method_package(files={"documents.mthds": NO_MAIN_PIPE_MTHDS}, manifest_toml=STUB_METHOD_MANIFEST_NO_MAIN_PIPE)
+ client = _build_client()
+ detail = _assert_input_422(client.post(PIPE_IO_PATH, json={"method_ref": STUB_METHOD_REF}), error_type=PIPE_NOT_FOUND_ERROR)
+ assert "main_pipe" in detail
+
@pytest.mark.parametrize(
("contents", "request_fields"),
[
diff --git a/api/tests/unit/test_validate_and_models_routes.py b/api/tests/unit/test_validate_and_models_routes.py
new file mode 100644
index 000000000..ce195f997
--- /dev/null
+++ b/api/tests/unit/test_validate_and_models_routes.py
@@ -0,0 +1,163 @@
+"""Smoke + validation tests for /validate and /models."""
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+from mthds.protocol.models import ModelCategory as MthdsModelCategory
+from pipelex.builder.operations.models_ops import ModelCategory
+from pipelex.interpreter_hub import get_library_manager
+from pytest_mock import MockerFixture
+
+from pipelex_api.exception_handlers import register_exception_handlers
+from pipelex_api.routes import router as api_router
+from tests.unit._constants import VALID_MTHDS
+
+
+def _build_client() -> TestClient:
+ app = FastAPI()
+ app.include_router(api_router, prefix="/v1")
+ register_exception_handlers(app)
+ return TestClient(app)
+
+
+class TestValidateAndModelsRoutes:
+ def test_validate_rejects_oversized_mthds(self):
+ client = _build_client()
+ oversized = "a" * (2 * 1024 * 1024) # 2 MiB > 1 MiB cap
+ response = client.post(
+ "/v1/validate",
+ json={"mthds_contents": [oversized]},
+ )
+ assert response.status_code == 422
+ assert response.headers["content-type"] == "application/problem+json"
+ assert response.json()["error_type"] == "ValidationError"
+
+ def test_validate_rejects_too_many_files(self):
+ client = _build_client()
+ response = client.post(
+ "/v1/validate",
+ json={"mthds_contents": [VALID_MTHDS] * 32},
+ )
+ assert response.status_code == 422
+ assert response.headers["content-type"] == "application/problem+json"
+ assert response.json()["error_type"] == "ValidationError"
+
+ def test_validate_invalid_mthds_returns_200_invalid_report(self):
+ # `/validate` is a diagnostic endpoint: an invalid bundle is a produced verdict, so it rides
+ # a 200 `InvalidReport` (discriminated on `is_valid: false`) — NOT a 422 problem document,
+ # and NOT the older `{success, mthds_contents, message}` 422 envelope. Invalid TOML reliably
+ # triggers a `ValidateBundleError` from the interpreter, which the route converts to the
+ # invalid arm. The verdict is carried by `is_valid`/`message`, not the status code.
+ client = _build_client()
+ response = client.post(
+ "/v1/validate",
+ json={"mthds_contents": ["this is not valid TOML !!!"]},
+ )
+ assert response.status_code == 200, response.text
+ assert response.headers["content-type"].startswith("application/json")
+ body = response.json()
+ assert body["is_valid"] is False
+ assert body["is_runnable"] is False
+ # The pipelex message is preserved (caller-facing under `_authors_caller_facing_message`) so
+ # the client gets the actual interpreter complaint, not a generic placeholder.
+ assert "TOML" in body["message"]
+ # The diagnostics list is non-empty on every invalid arm — the structured-info invariant is
+ # total. A raw TOML-syntax error is a parse-level failure the interpreter raises with no
+ # categorized error-data, but the shared builder's last-resort residual still emits one
+ # `blueprint_validation` item carrying the message (no `source` at this layer). A categorized
+ # failure (e.g. an invalid `main_pipe`) yields richer items; that path is pinned in
+ # test_validate_errors.py.
+ assert isinstance(body["validation_errors"], list)
+ assert body["validation_errors"], "an invalid verdict must carry a non-empty validation_errors[]"
+ assert body["validation_errors"][0]["category"] == "blueprint_validation"
+ assert body["validation_errors"][0]["message"]
+ # The valid arm's structural artifacts + the retired `success` extra are absent.
+ assert "bundle_blueprint" not in body
+ assert "success" not in body
+
+ # NOTE: the former `main_pipe` precondition on /validate is deleted (protocol alignment
+ # D2) — a bundle without `main_pipe` now answers 200 with `graph_spec=null`. The
+ # regression pin for that behavior (both backends) lives in `test_validate_envelope.py`.
+
+ @pytest.mark.parametrize("bad_type", ["not-a-real-category", ""])
+ def test_models_rejects_invalid_category(self, bad_type: str):
+ # The empty string is an explicitly-supplied (invalid) filter value, not an absent
+ # param — it must fail loudly like any other unknown category, not silently return
+ # the unfiltered deck.
+ client = _build_client()
+ response = client.get(f"/v1/models?type={bad_type}")
+ assert response.status_code == 422
+ assert response.headers["content-type"] == "application/problem+json"
+ assert response.json()["error_type"] == "InvalidModelCategory"
+
+ def test_models_rejects_repeated_type_param(self):
+ # D11: the protocol's `type` filter is a single plain value. The old route accepted
+ # repeated `?type=` values (list[str] Query) — that extension is dropped, and FastAPI
+ # would otherwise silently keep one of the values, so the rejection is explicit.
+ # Generic ValidationError (not InvalidModelCategory): both values are valid
+ # categories — what's wrong is the arity.
+ client = _build_client()
+ response = client.get("/v1/models?type=llm&type=extract")
+ assert response.status_code == 422
+ assert response.headers["content-type"] == "application/problem+json"
+ assert response.json()["error_type"] == "ValidationError"
+
+ def test_models_returns_protocol_deck(self):
+ # The deck is the protocol `ModelDeck` shape produced by `PipelexMTHDSProtocol.models`:
+ # a non-empty flat `models` list (regression for the silently-empty deck the old raw
+ # per-category payload caused in the SDK — F2) plus this implementation's routing
+ # extensions, keyed by category (the same alias name exists in several categories —
+ # a flat map would silently drop entries on collision). The old raw keys (`presets`
+ # by category, `success`) are gone. Every entry of the flat list is typed by one of the
+ # protocol's categories, `judgment` among them, and never by a value the protocol does not
+ # define; the extensions are keyed by the categories this runtime serves.
+ client = _build_client()
+ response = client.get("/v1/models")
+ assert response.status_code == 200, response.text
+ body = response.json()
+ assert body["models"], "the protocol deck must carry a non-empty models list"
+ first_model = body["models"][0]
+ assert first_model["name"]
+ assert first_model["type"]
+ assert {model["type"] for model in body["models"]} <= {category.value for category in MthdsModelCategory}
+ valid_categories = {category.value for category in ModelCategory}
+ assert set(body["aliases"]) <= valid_categories
+ assert all(isinstance(category_aliases, dict) for category_aliases in body["aliases"].values())
+ assert set(body["waterfalls"]) <= valid_categories
+ assert "presets" not in body
+ assert "success" not in body
+
+ def test_models_single_type_filter(self):
+ client = _build_client()
+ response = client.get("/v1/models?type=llm")
+ assert response.status_code == 200, response.text
+ body = response.json()
+ assert body["models"]
+ assert {model["type"] for model in body["models"]} == {"llm"}
+ assert set(body["aliases"]) <= {"llm"}
+
+ def test_empty_mthds_contents_rejected(self):
+ # `/validate` is the last route on the `mthds_contents` envelope (it is protocol-owned; the
+ # crate routes take `files[]`, whose empty-selector rejection is pinned in the pipe-io suite).
+ client = _build_client()
+ response = client.post("/v1/validate", json={"mthds_contents": []})
+ assert response.status_code == 422
+ assert response.headers["content-type"] == "application/problem+json"
+ assert response.json()["error_type"] == "ValidationError"
+
+ def test_validate_tears_down_validate_bundle_library_without_leaking(self, mocker: MockerFixture):
+ # validate_bundle opens a library and leaves it loaded + current on success (the D6 contract);
+ # /validate must own that teardown. Before the fix the route never tore it down, orphaning it on
+ # every successful call (the best-effort graph dry-run opens + tears down its OWN library via
+ # PipelexRunner, so that one was balanced). Assert the conservation property: every library opened
+ # during the request is torn down — open count == teardown count. A leak shows up as opens > teardowns.
+ library_manager = get_library_manager()
+ open_spy = mocker.spy(library_manager, "open_library")
+ teardown_spy = mocker.spy(library_manager, "teardown")
+
+ client = _build_client()
+ response = client.post("/v1/validate", json={"mthds_contents": [VALID_MTHDS]})
+
+ assert response.status_code == 200, response.text
+ assert open_spy.call_count >= 1
+ assert open_spy.call_count == teardown_spy.call_count
diff --git a/api/tests/unit/test_validate_caller_identity.py b/api/tests/unit/test_validate_caller_identity.py
index 030402258..2766d9765 100644
--- a/api/tests/unit/test_validate_caller_identity.py
+++ b/api/tests/unit/test_validate_caller_identity.py
@@ -1,4 +1,4 @@
-"""`/validate` and `/build/runner` hand the runtime the caller their validation sweep is done for.
+"""`/validate` hands the runtime the caller its validation sweep is done for.
A validation is not a run, but its dry runs and its `pipe_dry_run` event are still telemetry, and
the runtime attributes them to the `CallerIdentity` it is handed. Every failure on this path is
@@ -6,7 +6,7 @@ SILENT by construction: a keyword left out of `ApiRunner(...)` or `validate_bund
answers 200, and the only symptom is that the sweep lands in PostHog under the deployment's
constant id instead of the caller. So these tests read the caller off the call the runtime
actually received — a recording validator for the dispatched `/validate`, the real in-process
-entry points (wrapped, not replaced) for the direct path and for `/build/runner`.
+entry point (wrapped, not replaced) for the direct path.
The user comes from `request.state.user`, which the auth layer sets; a tiny middleware stands in
for it here. The groups come from the body and are refused at the wire exactly as on a run.
@@ -20,7 +20,6 @@ import pytest
from fastapi import FastAPI, Request, Response
from fastapi.testclient import TestClient
from pipelex.base_exceptions import ErrorReport, ValidationErrorCategory, ValidationErrorItem
-from pipelex.pipeline.validate_bundle import validate_bundle
from pipelex.pipeline.validate_in_process import validate_bundles_in_process
from pipelex.plugins.bundle_validator_registry import BundleValidatorRegistry
from pipelex.system.caller_identity import CallerIdentity
@@ -34,7 +33,6 @@ from pipelex_api.security import RequestUser
from tests.unit._constants import STUB_METHOD_ADDRESS, VALID_MTHDS
_PIPELINE_NS = "pipelex_api.routes.pipelex.pipeline"
-_RUNNER_NS = "pipelex_api.routes.pipelex.build.runner"
_DIRECT_VALIDATOR_NS = "pipelex.pipeline.direct_bundle_validator"
_MODE = "temporal"
_USER_ID = "user_42"
@@ -191,42 +189,3 @@ class TestValidationCallerIdentity:
assert "analytics_groups" in body["detail"]
assert "allow_signatures" in body["detail"]
assert validator.caller_identities == []
-
- def test_build_runner_sweeps_for_the_caller(self, mocker: MockerFixture) -> None:
- sweep = mocker.patch(f"{_RUNNER_NS}.validate_bundle", wraps=validate_bundle)
- client = _build_client(user_id=_USER_ID)
-
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo", "analytics_groups": _GROUPS},
- )
-
- assert response.status_code == 200, response.text
- assert response.json()["is_valid"] is True
- assert sweep.call_count == 1
- assert _caller_of(sweep.call_args) == CallerIdentity(user_id=_USER_ID, extras=_GROUPS)
-
- def test_build_runner_without_groups_or_user_states_the_single_tenant_caller(self, mocker: MockerFixture) -> None:
- sweep = mocker.patch(f"{_RUNNER_NS}.validate_bundle", wraps=validate_bundle)
- client = _build_client(user_id=None)
-
- response = client.post("/v1/build/runner", json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo"})
-
- assert response.status_code == 200, response.text
- assert _caller_of(sweep.call_args) == CallerIdentity(user_id=SINGLE_TENANT_USER_ID, extras={})
-
- @pytest.mark.parametrize("bad_groups", _BAD_GROUPS)
- def test_build_runner_refuses_malformed_groups_with_a_422_naming_the_field(self, mocker: MockerFixture, bad_groups: Any) -> None:
- sweep = mocker.patch(f"{_RUNNER_NS}.validate_bundle", wraps=validate_bundle)
- client = _build_client(user_id=_USER_ID)
-
- response = client.post(
- "/v1/build/runner",
- json={"files": [{"content": VALID_MTHDS}], "pipe_ref": "smoke.echo", "analytics_groups": bad_groups},
- )
-
- assert response.status_code == 422, response.text
- body = response.json()
- assert body["error_type"] == "InvalidAnalyticsGroups"
- assert "analytics_groups" in body["detail"]
- assert sweep.call_count == 0
diff --git a/api/tests/unit/test_validate_default_pipe_ref.py b/api/tests/unit/test_validate_default_pipe_ref.py
index 6caf405c1..158ad78ff 100644
--- a/api/tests/unit/test_validate_default_pipe_ref.py
+++ b/api/tests/unit/test_validate_default_pipe_ref.py
@@ -58,7 +58,7 @@ class TestValidateDefaultPipeRef:
def test_several_declared_main_pipes_report_the_run_default_not_null(self):
# The field states the RUN default (`select_primary_blueprint`: first blueprint declaring
- # `main_pipe`), NOT the build routes' stricter rule that 422s on several. `/execute` and
+ # `main_pipe`), NOT `/pipe-io`'s stricter rule that 422s on several. `/execute` and
# `/start` run this closure happily, so reporting null would make a consumer refuse to
# prepare a method the server would run.
client = _build_client()
diff --git a/docs/api-server/codegen.md b/docs/api-server/codegen.md
index 0f916f3f6..29f012059 100644
--- a/docs/api-server/codegen.md
+++ b/docs/api-server/codegen.md
@@ -59,7 +59,7 @@ Resolution is a first-class language operation alongside validation: the closure
Resolves the closure exactly like `/resolve`, then projects the crate through the two explicit axes:
-- `kind` (string, required): what to project. Served: `types` (the crate's concept set as typed models). Membership follows the **trust chain** — a kind is served here exactly when its artifacts are stamped and locked, which is the promise the valid arm makes by carrying a `lock`. Input templates are user-editable scaffolds, never stamped or locked, so they cannot make that promise: they ride [`POST /v1/build/inputs`](pipe-builder.md) instead and `inputs` is deliberately **not** a kind here.
+- `kind` (string, required): what to project. Served: `types` (the crate's concept set as typed models). Membership follows the **trust chain** — a kind is served here exactly when its artifacts are stamped and locked, which is the promise the valid arm makes by carrying a `lock`. Input templates are user-editable scaffolds, never stamped or locked, so they cannot make that promise: a client projects one from the input form [`POST /v1/pipe-io`](pipe-io.md) serves, and `inputs` is deliberately **not** a kind here.
- `target` (string, required): for whom. `ts-zod` (zod schemas + inferred types), `python-pydantic` (self-contained pydantic models), or `python-structures` (runtime `StructuredContent` classes, for a Pipelex host).
- `pipe_ref` (string, optional): pipe selector for future per-pipe kinds — not accepted for `types` (request-shape `422`).
diff --git a/docs/api-server/error-responses.md b/docs/api-server/error-responses.md
index f376724d3..2dafff46b 100644
--- a/docs/api-server/error-responses.md
+++ b/docs/api-server/error-responses.md
@@ -35,11 +35,11 @@ A failure response looks like this:
- `error_category` — finer classification when the originating error provides one (currently used by inference errors — see [`InferenceErrorCategory`](https://docs.pipelex.com/latest/errors/) upstream).
- `user_action` — structured suggestion of what the caller should do next, when the error can author one.
- `model`, `provider`, `provider_metadata` — populated when the failure originated in an upstream inference call. **Stripped under STRICT disclosure** (see [Disclosure modes](#disclosure-modes)).
-- `validation_errors` — structured per-error list carried by a `ValidateBundleError`. On the run routes (`/execute`, `/start`) it rides this 422 problem document; on the diagnostic routes (`/validate`, `/resolve`, `/codegen`, `/build/{inputs,output,runner}`) it rides the **200 `is_valid: false`** body instead (see [Pipe Validate](pipe-validate.md)). See [Structured validation errors](#structured-validation-errors).
+- `validation_errors` — structured per-error list carried by a `ValidateBundleError`. On the run routes (`/execute`, `/start`) it rides this 422 problem document; on the diagnostic routes (`/validate`, `/resolve`, `/codegen`, `/pipe-io`) it rides the **200 `is_valid: false`** body instead (see [Pipe Validate](pipe-validate.md)). See [Structured validation errors](#structured-validation-errors).
## Structured validation errors
-When a bundle fails validation, the `ValidateBundleError` carries a `validation_errors` array — the per-error diagnostics an editor maps to per-line problems. Where it surfaces depends on the endpoint: on the diagnostic routes (`/validate`, `/resolve`, `/codegen`, `/build/{inputs,output,runner}`) it rides the **200 `is_valid: false`** body (the diagnostic-endpoint contract — see [Pipe Validate](pipe-validate.md)); on the run routes (`/execute`, `/start`) — where an invalid bundle means the run cannot proceed — it rides the **422** problem document alongside the single human-readable `detail`. Built by pipelex's one shared builder, the items are identical wherever they appear (and to the agent CLI's). Each item is one categorized validation failure:
+When a bundle fails validation, the `ValidateBundleError` carries a `validation_errors` array — the per-error diagnostics an editor maps to per-line problems. Where it surfaces depends on the endpoint: on the diagnostic routes (`/validate`, `/resolve`, `/codegen`, `/pipe-io`) it rides the **200 `is_valid: false`** body (the diagnostic-endpoint contract — see [Pipe Validate](pipe-validate.md)); on the run routes (`/execute`, `/start`) — where an invalid bundle means the run cannot proceed — it rides the **422** problem document alongside the single human-readable `detail`. Built by pipelex's one shared builder, the items are identical wherever they appear (and to the agent CLI's). Each item is one categorized validation failure:
| Field | Meaning |
|---|---|
@@ -169,15 +169,15 @@ Every route that takes a JSON body refuses one whose arrays and objects nest mor
The run routes read their body as plain JSON before anything else, and two refusals come from that step, both **422**. A body that is not valid UTF-8, not valid JSON, nested too deeply, or not a JSON object answers `InvalidJSON`. A body carrying, in any object at any depth, a key named `__class__` or `__module__`, or a key starting with `__kajson`, answers `ReservedObjectKey`, and the `detail` names the key, cut short when it is long. Those are the keys a kajson decoder reads as an order to import and instantiate a class, so the runner refuses them rather than carry them as data. Only keys are checked: a string value that mentions `__class__` is accepted.
-A pipe selection the server cannot make is an input **422** named for its failure, never the generic `ValidationError`: `EntryPipeNotFoundError` when the selector names no pipe, and `EntryPipeAmbiguousError` when a bare code matches pipes in several domains. The run routes answer an unknown or ambiguous `pipe_code` that way. `POST /v1/pipe-io` and the `/v1/build/*` routes answer every selection refusal that way, counting a request with no `pipe_ref` over a method that declares no `main_pipe` as `EntryPipeNotFoundError` and one over a method whose domains declare several as `EntryPipeAmbiguousError` (see [Pipe I/O](pipe-io.md#pipe-selection)).
+A pipe selection the server cannot make is an input **422** named for its failure, never the generic `ValidationError`: `EntryPipeNotFoundError` when the selector names no pipe, and `EntryPipeAmbiguousError` when a bare code matches pipes in several domains. The run routes answer an unknown or ambiguous `pipe_code` that way. `POST /v1/pipe-io` answers every selection refusal that way, counting a request with no `pipe_ref` over a method that declares no `main_pipe` as `EntryPipeNotFoundError` and one over a method whose domains declare several as `EntryPipeAmbiguousError` (see [Pipe I/O](pipe-io.md#pipe-selection)).
-The run routes' extension fields are validated before any method is fetched or any library loaded, and a failure there is a **422** whose `error_type` names the field that failed: `InvalidCallbackUrls` for `callback_urls`, `InvalidStorageScope` for `storage_scope`, `InvalidReadScope` for `read_scope`, `InvalidAnalyticsGroups` for `analytics_groups` (see [Pipe Run](pipe-run.md#host-supplied-run-context-storage_scope-and-analytics_groups)). A storage scope that does not lie under the run's read scope, whether either was sent or defaulted, is an `InvalidReadScope` as well. A failure on another extension field, or on more than one at once, carries the generic `ValidationError`, and the `detail` names every field that failed. `POST /v1/validate` and `POST /v1/build/runner` take `analytics_groups` too and classify it the same way: a body failing on that field alone answers `InvalidAnalyticsGroups`, and one failing on it and another field answers `ValidationError`.
+The run routes' extension fields are validated before any method is fetched or any library loaded, and a failure there is a **422** whose `error_type` names the field that failed: `InvalidCallbackUrls` for `callback_urls`, `InvalidStorageScope` for `storage_scope`, `InvalidReadScope` for `read_scope`, `InvalidAnalyticsGroups` for `analytics_groups` (see [Pipe Run](pipe-run.md#host-supplied-run-context-storage_scope-and-analytics_groups)). A storage scope that does not lie under the run's read scope, whether either was sent or defaulted, is an `InvalidReadScope` as well. A failure on another extension field, or on more than one at once, carries the generic `ValidationError`, and the `detail` names every field that failed. `POST /v1/validate` takes `analytics_groups` too and classifies it the same way: a body failing on that field alone answers `InvalidAnalyticsGroups`, and one failing on it and another field answers `ValidationError`.
The HTTP status is the source of truth for success vs failure — there is no `success: true/false` field anywhere in the envelope.
Which statuses a given route can actually produce is documented per operation in the [committed OpenAPI artifact](openapi/pipelex-api.openapi.yaml), each as an `application/problem+json` `ProblemDocument`.
-**What is *not* an error status:** an invalid `.mthds` bundle. On the diagnostic routes — `/validate`, `/resolve`, `/codegen`, and `/build/{inputs,output,runner}` — a bundle that fails validation is the *successful product* of the call, so it rides a **200** discriminated on `is_valid: false`, carrying the same `validation_errors[]` described above. Non-2xx on those routes is reserved for *no verdict could be produced*. See [Pipe Validate](pipe-validate.md).
+**What is *not* an error status:** an invalid `.mthds` bundle. On the diagnostic routes — `/validate`, `/resolve`, `/codegen`, and `/pipe-io` — a bundle that fails validation is the *successful product* of the call, so it rides a **200** discriminated on `is_valid: false`, carrying the same `validation_errors[]` described above. Non-2xx on those routes is reserved for *no verdict could be produced*. See [Pipe Validate](pipe-validate.md).
## Disclosure modes
diff --git a/docs/api-server/index.md b/docs/api-server/index.md
index f14d64335..a57ca87fc 100644
--- a/docs/api-server/index.md
+++ b/docs/api-server/index.md
@@ -16,11 +16,11 @@ This server is the source-available reference implementation of the **[MTHDS Pro
```
MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API
-(the standard) (protocol + build tooling) (+ durable runs, catalog, account)
+(the standard) (protocol + extensions) (+ durable runs, catalog, account)
```
- **MTHDS Protocol** — five routes: `POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version`. Tagged `x-mthds-protocol: true` in the [committed OpenAPI artifact](openapi/pipelex-api.openapi.yaml), and **only** those five — the flag is how a conformance suite or a third-party runner extracts the portable subset.
-- **Pipelex API (this server)** — the protocol verbatim, plus the Pipelex extensions: resolve and codegen (`/resolve`, `/codegen`), pipe I/O (`/pipe-io`), build tooling (`/build/*`), and editor tooling (`/lint`, `/format`). `/upload` and `/resolve-storage-url` were non-contract convenience routes and have been **removed** — see [Storage Transport](storage-transport.md) for where they went and why.
+- **Pipelex API (this server)** — the protocol verbatim, plus the Pipelex extensions: resolve and codegen (`/resolve`, `/codegen`), pipe I/O (`/pipe-io`), and editor tooling (`/lint`, `/format`). `/upload` and `/resolve-storage-url` were non-contract convenience routes and have been **removed** — see [Storage Transport](storage-transport.md) for where they went and why.
- **Pipelex hosted API** (`api.pipelex.com/v1`) — everything here, same shapes, plus durable runs, the method catalog, and account management.
All routes are served under the `/v1` base path (clients compose `{base}/v1/{endpoint}`).
@@ -32,7 +32,7 @@ The API currently allows you to:
1. **Run** any Pipelex pipeline with flexible inputs (sync or async)
2. **Validate** any Pipelex pipeline to ensure correctness
3. **Resolve** a library closure into its normalized crate, and **codegen** typed artifacts from it (TypeScript/Zod, Pydantic, Pipelex structures)
-4. **Build** pipeline components — generate input schemas, output representations, runner code, concepts, and pipe specs
+4. **Describe** a method's inputs and output — its pipe I/O contracts, input form and output form, from which a client projects an inputs template
5. **Lint and format** single `.mthds` files for editor workflows
6. **List** available model presets and configurations
@@ -180,20 +180,9 @@ Lint and format single `.mthds` files without loading or executing a pipeline.
[Learn more →](mthds-tools.md)
-### Pipe Builder
-Generate input templates, output representations, and runner code for one pipe of a library closure. All three take the same closure selector as `/v1/resolve` and `/v1/codegen` (inline `files[]` XOR `method_ref`), plus an optional qualified `pipe_ref` defaulting to the fetched package manifest's `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`.
+### Models
+List the models this runner can route to.
-- `POST /v1/build/inputs` — Generate an example inputs template for a pipe (JSON or TOML)
-- `POST /v1/build/output` — Generate an output representation (schema, JSON, or Python)
-- `POST /v1/build/runner` — Generate Python runner code for a pipe
-
-[Learn more →](pipe-builder.md)
-
-### Agent
-Tools for AI agents building pipelines programmatically.
-
-- `POST /v1/build/concept` — Convert a JSON concept spec to TOML
-- `POST /v1/build/pipe-spec` — Convert a JSON pipe spec to TOML
- `GET /v1/models` — The protocol model deck this runner routes to (flat `models` list, plus category-keyed `aliases`/`waterfalls` routing extensions); optional single `?type=` category filter
### Uploader (removed)
diff --git a/docs/api-server/openapi/pipelex-api.openapi.yaml b/docs/api-server/openapi/pipelex-api.openapi.yaml
index edb78691c..576d3938f 100644
--- a/docs/api-server/openapi/pipelex-api.openapi.yaml
+++ b/docs/api-server/openapi/pipelex-api.openapi.yaml
@@ -5,9 +5,9 @@ info:
summary: The source-available Pipelex runner — implements MTHDS Protocol v0.7.0.
description: 'This server implements the [MTHDS Protocol](https://mthds.ai) v0.7.0 (`POST /execute`, `POST /start`, `POST
/validate`, `GET /models`, `GET /version` — marked `x-mthds-protocol: true`) plus the Pipelex API extensions: resolve
- and codegen (`/resolve`, `/codegen`), build tooling (`/build/*`), and editor tooling (`/lint`, `/format`). Contract layering:
- MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API. All endpoints are served under the `/v1` base path; every
- error is an RFC 7807 `application/problem+json` problem document, documented per operation as a `ProblemDocument`.'
+ and codegen (`/resolve`, `/codegen`), the pipe I/O route (`/pipe-io`), and editor tooling (`/lint`, `/format`). Contract
+ layering: MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API. All endpoints are served under the `/v1` base
+ path; every error is an RFC 7807 `application/problem+json` problem document, documented per operation as a `ProblemDocument`.'
license:
name: Elastic License 2.0
identifier: Elastic-2.0
@@ -67,307 +67,6 @@ paths:
schema:
$ref: '#/components/schemas/ProblemDocument'
x-mthds-protocol: true
- /v1/build/inputs:
- post:
- tags:
- - build
- summary: Build Inputs
- description: "Generate an example inputs template for a pipe (the inputs projection, per pipe).\n\nRides the **same\
- \ static core** as `POST /resolve` and `POST /codegen`: the closure is resolved to\nits normalized crate, the requested\
- \ pipe is read live from the library that leaves loaded, and its\n*declared* inputs are rendered — the exact projection\
- \ `pipelex codegen inputs` writes, on both of\nits axes (`format`, `explicit`).\n\nStatic is the point: a template\
- \ is a read of the pipe's declared IO, so there is **no dry-run\nsweep** here (that is `/validate`'s vocabulary, and\
- \ `/build/runner`'s need). A valid verdict says\nthe closure is structurally sound and the template matches what the\
- \ pipe declares — it is *not* a\npromise the pipe runs. Ask `/validate` for that.\n\nThis projection is deliberately\
- \ **not** a `/codegen` kind: the templates are user-editable\nscaffolds, never stamped or locked, so they cannot ride\
- \ the trust chain `/codegen`'s valid arm\npromises (see `CodegenRouteKind`).\n\nResponse contract (the `/validate`\
- \ discipline):\n\n- **Valid verdict (200, `is_valid: true`):** the template, in `inputs` or `inputs_toml` per `format`.\n\
- - **Invalid verdict (200, `is_valid: false`):** the closure could not be parsed, loaded, or\n validated — `validation_errors[]`\
- \ from pipelex's one shared builder; no template exists.\n- **No verdict (non-2xx):** a selection refusal — an unknown\
- \ pipe ref, or an omitted `pipe_ref` that\n nothing defaults (no fetched-manifest `main_pipe`, and a closure declaring\
- \ no — or several —\n `main_pipe`) — is an input 422 whose `error_type` is `EntryPipeNotFoundError` or\n `EntryPipeAmbiguousError`;\
- \ a malformed closure selector is a request-shape 422 problem+json; a\n registry-form `method_ref` is a 501 until\
- \ server-side method-registry resolution exists.\n\nAn omitted `pipe_ref` defaults the way a run by address does:\
- \ to the fetched package manifest's\n`main_pipe` on a `method_ref` request, else to the closure's own declared `main_pipe`."
- operationId: build_inputs_v1_build_inputs_post
- requestBody:
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildInputsRequest'
- required: true
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- oneOf:
- - $ref: '#/components/schemas/BuildInputsValidReport'
- - $ref: '#/components/schemas/CrateInvalidReport'
- title: Response Build Inputs V1 Build Inputs Post
- discriminator:
- propertyName: is_valid
- mapping:
- 'True': '#/components/schemas/BuildInputsValidReport'
- 'False': '#/components/schemas/CrateInvalidReport'
- '401':
- description: Missing or invalid bearer token. Only reachable when the deployment enables auth (`AUTH_MODE=api_key`
- or `AUTH_MODE=jwt`).
- headers:
- WWW-Authenticate:
- description: Authentication challenge — always `Bearer`.
- schema:
- type: string
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '413':
- description: Request body exceeds the deployment's size limit (`MAX_REQUEST_BODY_MIB`, 100 MiB by default).
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '422':
- description: 'The request could not be processed: a malformed body, a field failing validation, or an `input`-domain
- pipelex error (a `.mthds` bundle the caller must fix). Note that on the diagnostic routes an *invalid bundle*
- is a **200** verdict, not a 422 — see each route''s response contract.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '500':
- description: A `config`-domain or `runtime`-domain failure the caller cannot fix (a missing env var, a bad TOML
- override, a backend fault), or an unclassified error sanitized by the catch-all handler. Report the `request_id`.
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '404':
- description: '`MethodPackageNotFoundError` — the `method_ref` repository was fetched, but no package in it matches
- the requested address by manifest identity. The message lists the packages the repository does contain.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '501':
- description: '`MethodRefNotSupported` — the request selected its closure by a **registry-form** `method_ref` (not
- a `github.com/...` address), and no server-side method registry resolves those yet. Use an address-form reference
- or submit inline `files[]` instead.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- /v1/build/output:
- post:
- tags:
- - build
- summary: Build Output
- description: "Generate an example output representation for a pipe (the output projection, per pipe).\n\nRides the **same\
- \ static core** as `POST /resolve`, `POST /codegen` and `POST /build/inputs`: the\nclosure is resolved to its normalized\
- \ crate, the requested pipe is read live from the library that\nleaves loaded, and its *declared* output is rendered.\n\
- \nStatic is the point: the representation is a read of the pipe's declared IO, so there is **no\ndry-run sweep** here.\
- \ A valid verdict says the closure is structurally sound and the shape matches\nwhat the pipe declares — it is *not*\
- \ a promise the pipe runs. Ask `/validate` for that.\n\nResponse contract (the `/validate` discipline):\n\n- **Valid\
- \ verdict (200, `is_valid: true`):** the representation, in `output` or `output_python` per `format`.\n- **Invalid\
- \ verdict (200, `is_valid: false`):** the closure could not be parsed, loaded, or\n validated — `validation_errors[]`;\
- \ no representation exists.\n- **No verdict (non-2xx):** a selection refusal — an unknown pipe ref, or an omitted\
- \ `pipe_ref` that\n nothing defaults (no fetched-manifest `main_pipe`, and a closure declaring no — or several —\n\
- \ `main_pipe`) — is an input 422 whose `error_type` is `EntryPipeNotFoundError` or\n `EntryPipeAmbiguousError`;\
- \ a pipe whose `native.Anything` output has no determinable shape, or a\n malformed closure selector, is a request-shape\
- \ 422 problem+json; a registry-form `method_ref` is a 501 until server-side\n method-registry resolution exists.\n\
- \nAn omitted `pipe_ref` defaults the way a run by address does: to the fetched package manifest's\n`main_pipe` on\
- \ a `method_ref` request, else to the closure's own declared `main_pipe`."
- operationId: build_output_v1_build_output_post
- requestBody:
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildOutputRequest'
- required: true
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- oneOf:
- - $ref: '#/components/schemas/BuildOutputValidReport'
- - $ref: '#/components/schemas/CrateInvalidReport'
- title: Response Build Output V1 Build Output Post
- discriminator:
- propertyName: is_valid
- mapping:
- 'True': '#/components/schemas/BuildOutputValidReport'
- 'False': '#/components/schemas/CrateInvalidReport'
- '401':
- description: Missing or invalid bearer token. Only reachable when the deployment enables auth (`AUTH_MODE=api_key`
- or `AUTH_MODE=jwt`).
- headers:
- WWW-Authenticate:
- description: Authentication challenge — always `Bearer`.
- schema:
- type: string
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '413':
- description: Request body exceeds the deployment's size limit (`MAX_REQUEST_BODY_MIB`, 100 MiB by default).
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '422':
- description: 'The request could not be processed: a malformed body, a field failing validation, or an `input`-domain
- pipelex error (a `.mthds` bundle the caller must fix). Note that on the diagnostic routes an *invalid bundle*
- is a **200** verdict, not a 422 — see each route''s response contract.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '500':
- description: A `config`-domain or `runtime`-domain failure the caller cannot fix (a missing env var, a bad TOML
- override, a backend fault), or an unclassified error sanitized by the catch-all handler. Report the `request_id`.
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '404':
- description: '`MethodPackageNotFoundError` — the `method_ref` repository was fetched, but no package in it matches
- the requested address by manifest identity. The message lists the packages the repository does contain.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '501':
- description: '`MethodRefNotSupported` — the request selected its closure by a **registry-form** `method_ref` (not
- a `github.com/...` address), and no server-side method registry resolves those yet. Use an address-form reference
- or submit inline `files[]` instead.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- /v1/build/runner:
- post:
- tags:
- - build
- summary: Build Runner
- description: 'Generate a Python runner script for a pipe, riding the codegen types projection.
-
-
- The one `/build/*` projection that is **not** static: a runner script is a promise the pipe can
-
- actually run, so this route keeps the dry-run sweep its siblings dropped (and with it
-
- `allow_signatures`, which only ever parameterized that sweep). `validate_bundle` opens a single
-
- library, loads the closure, sweeps, and on success leaves the library loaded + current; on failure
-
- it tears it down itself. On the success path the crate is read from that library, the
-
- `python-structures` projection is emitted and stamped, and the runner script is generated with the
-
- **emitted** class names — the same flow as a local `pipelex build runner`.
-
-
- The sweep is scoped to the requested pipe, so unrelated broken siblings do not block a good pipe.
-
- When `pipe_ref` is omitted the scope is not settled before the closure loads, so the whole closure
-
- is swept and the pipe then defaults the way a run by address does — to the fetched package
-
- manifest''s `main_pipe` on a `method_ref` request, else to the closure''s own declared `main_pipe` —
-
- a stricter verdict, and the honest one for a caller who did not say which pipe they meant.
-
-
- Response contract (the `/validate` discipline): an invalid closure — including a failed dry-run of
-
- the requested pipe — is a produced verdict: a **200** `is_valid: false` with the structured
-
- `validation_errors[]`. Non-2xx is reserved for no-verdict conditions: a selection refusal (an
-
- unknown pipe ref, an omitted `pipe_ref` that nothing defaults — no fetched-manifest `main_pipe`,
-
- and a closure declaring no, or several, `main_pipe`), an input 422 whose `error_type` is
-
- `EntryPipeNotFoundError` or `EntryPipeAmbiguousError`; a request-shape 422 for a requested pipe
-
- whose cross-package dependencies are absent from the request; a 501 for a registry-form `method_ref`, auth, server
-
- fault — RFC 7807 via the global handlers.'
- operationId: build_runner_v1_build_runner_post
- requestBody:
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildRunnerRequest'
- required: true
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- oneOf:
- - $ref: '#/components/schemas/BuildRunnerValidReport'
- - $ref: '#/components/schemas/CrateInvalidReport'
- title: Response Build Runner V1 Build Runner Post
- discriminator:
- propertyName: is_valid
- mapping:
- 'True': '#/components/schemas/BuildRunnerValidReport'
- 'False': '#/components/schemas/CrateInvalidReport'
- '401':
- description: Missing or invalid bearer token. Only reachable when the deployment enables auth (`AUTH_MODE=api_key`
- or `AUTH_MODE=jwt`).
- headers:
- WWW-Authenticate:
- description: Authentication challenge — always `Bearer`.
- schema:
- type: string
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '413':
- description: Request body exceeds the deployment's size limit (`MAX_REQUEST_BODY_MIB`, 100 MiB by default).
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '422':
- description: 'The request could not be processed: a malformed body, a field failing validation, or an `input`-domain
- pipelex error (a `.mthds` bundle the caller must fix). Note that on the diagnostic routes an *invalid bundle*
- is a **200** verdict, not a 422 — see each route''s response contract.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '500':
- description: A `config`-domain or `runtime`-domain failure the caller cannot fix (a missing env var, a bad TOML
- override, a backend fault), or an unclassified error sanitized by the catch-all handler. Report the `request_id`.
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '404':
- description: '`MethodPackageNotFoundError` — the `method_ref` repository was fetched, but no package in it matches
- the requested address by manifest identity. The message lists the packages the repository does contain.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '501':
- description: '`MethodRefNotSupported` — the request selected its closure by a **registry-form** `method_ref` (not
- a `github.com/...` address), and no server-side method registry resolves those yet. Use an address-form reference
- or submit inline `files[]` instead.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
/v1/execute:
post:
tags:
@@ -1449,159 +1148,20 @@ paths:
schema:
$ref: '#/components/schemas/ProblemDocument'
x-mthds-protocol: true
- /v1/build/concept:
- post:
+ /:
+ get:
tags:
- - agent
- summary: Build Concept
- description: 'Convert a JSON concept spec to TOML format.
-
-
- A malformed spec surfaces as a Pydantic `ValidationError` — an API-owned
-
- 422. Pipelex domain failures propagate untouched to the global
-
- `PipelexError` handler in `pipelex_api.exception_handlers`.
-
-
- A malformed shape — a non-mapping `structure` (`{"structure": "string"}`)
-
- or a `structure` field that is neither a string nor a mapping
+ - health
+ summary: Service identity banner
+ description: 'Identify the service answering on this origin. No auth required.
- (`{"structure": {"f": 42}}`) — is refused by `parse_concept_spec` itself
- with a typed `ConceptSpecError`, which that same handler answers. Nothing
+ A human-facing banner for someone who lands on the bare origin — not a
- broader is caught here: a bare exception from pipelex is the signal of a
+ liveness probe (that is `GET /health`) and not the protocol handshake
- real programming bug, and a route-level catch would mask it.'
- operationId: build_concept_v1_build_concept_post
- requestBody:
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildConceptRequest'
- required: true
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildConceptResponse'
- '401':
- description: Missing or invalid bearer token. Only reachable when the deployment enables auth (`AUTH_MODE=api_key`
- or `AUTH_MODE=jwt`).
- headers:
- WWW-Authenticate:
- description: Authentication challenge — always `Bearer`.
- schema:
- type: string
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '413':
- description: Request body exceeds the deployment's size limit (`MAX_REQUEST_BODY_MIB`, 100 MiB by default).
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '422':
- description: 'The request could not be processed: a malformed body, a field failing validation, or an `input`-domain
- pipelex error (a `.mthds` bundle the caller must fix). Note that on the diagnostic routes an *invalid bundle*
- is a **200** verdict, not a 422 — see each route''s response contract.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '500':
- description: A `config`-domain or `runtime`-domain failure the caller cannot fix (a missing env var, a bad TOML
- override, a backend fault), or an unclassified error sanitized by the catch-all handler. Report the `request_id`.
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- /v1/build/pipe-spec:
- post:
- tags:
- - agent
- summary: Build Pipe Spec
- description: 'Convert a JSON pipe spec to TOML format.
-
-
- Two caller-mistake paths surface as a 422 here: a `ValidationError` from
-
- Pydantic when the spec shape doesn''t match the chosen pipe type, and a
-
- `ValueError` from `parse_pipe_spec` when `pipe_type` is not one of the
-
- known pipe types (documented in `parse_pipe_spec`''s docstring; raised at
-
- exactly one site). Pipelex domain failures propagate untouched to the
-
- global `PipelexError` handler in `pipelex_api.exception_handlers`.'
- operationId: build_pipe_spec_v1_build_pipe_spec_post
- requestBody:
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildPipeSpecRequest'
- required: true
- responses:
- '200':
- description: Successful Response
- content:
- application/json:
- schema:
- $ref: '#/components/schemas/BuildPipeSpecResponse'
- '401':
- description: Missing or invalid bearer token. Only reachable when the deployment enables auth (`AUTH_MODE=api_key`
- or `AUTH_MODE=jwt`).
- headers:
- WWW-Authenticate:
- description: Authentication challenge — always `Bearer`.
- schema:
- type: string
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '413':
- description: Request body exceeds the deployment's size limit (`MAX_REQUEST_BODY_MIB`, 100 MiB by default).
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '422':
- description: 'The request could not be processed: a malformed body, a field failing validation, or an `input`-domain
- pipelex error (a `.mthds` bundle the caller must fix). Note that on the diagnostic routes an *invalid bundle*
- is a **200** verdict, not a 422 — see each route''s response contract.'
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- '500':
- description: A `config`-domain or `runtime`-domain failure the caller cannot fix (a missing env var, a bad TOML
- override, a backend fault), or an unclassified error sanitized by the catch-all handler. Report the `request_id`.
- content:
- application/problem+json:
- schema:
- $ref: '#/components/schemas/ProblemDocument'
- /:
- get:
- tags:
- - health
- summary: Service identity banner
- description: 'Identify the service answering on this origin. No auth required.
-
-
- A human-facing banner for someone who lands on the bare origin — not a
-
- liveness probe (that is `GET /health`) and not the protocol handshake
-
- (`GET /v1/version`). It reads nothing and can only succeed.'
- operationId: root__get
+ (`GET /v1/version`). It reads nothing and can only succeed.'
+ operationId: root__get
responses:
'200':
description: Successful Response
@@ -1686,505 +1246,128 @@ components:
anyOf:
- type: string
- type: 'null'
- title: Title
- concept_ref:
- anyOf:
- - type: string
- - type: 'null'
- title: Concept Ref
- refines:
- anyOf:
- - items:
- type: string
- type: array
- - type: 'null'
- title: Refines
- description:
- anyOf:
- - type: string
- - type: 'null'
- title: Description
- required:
- type: boolean
- title: Required
- presence:
- anyOf:
- - $ref: '#/components/schemas/PresenceMarker'
- - type: 'null'
- gating:
- anyOf:
- - type: boolean
- - type: 'null'
- title: Gating
- default_value:
- title: Default Value
- examples:
- anyOf:
- - items: {}
- type: array
- - type: 'null'
- title: Examples
- hints:
- anyOf:
- - additionalProperties:
- type: string
- type: object
- - type: 'null'
- title: Hints
- kind:
- type: string
- const: boolean
- title: Kind
- default: boolean
- name:
- type: string
- title: Name
- additionalProperties: false
- type: object
- required:
- - required
- - name
- title: BooleanField
- description: '`boolean` at a named position.'
- BooleanItem:
- properties:
- title:
- anyOf:
- - type: string
- - type: 'null'
- title: Title
- concept_ref:
- anyOf:
- - type: string
- - type: 'null'
- title: Concept Ref
- refines:
- anyOf:
- - items:
- type: string
- type: array
- - type: 'null'
- title: Refines
- description:
- anyOf:
- - type: string
- - type: 'null'
- title: Description
- required:
- type: boolean
- title: Required
- presence:
- anyOf:
- - $ref: '#/components/schemas/PresenceMarker'
- - type: 'null'
- gating:
- anyOf:
- - type: boolean
- - type: 'null'
- title: Gating
- default_value:
- title: Default Value
- examples:
- anyOf:
- - items: {}
- type: array
- - type: 'null'
- title: Examples
- hints:
- anyOf:
- - additionalProperties:
- type: string
- type: object
- - type: 'null'
- title: Hints
- kind:
- type: string
- const: boolean
- title: Kind
- default: boolean
- additionalProperties: false
- type: object
- required:
- - required
- title: BooleanItem
- description: '`boolean` — true or false.'
- BuildConceptRequest:
- properties:
- spec:
- additionalProperties: true
- type: object
- title: Spec
- description: JSON concept specification.
- type: object
- required:
- - spec
- title: BuildConceptRequest
- BuildConceptResponse:
- properties:
- success:
- type: boolean
- title: Success
- description: Whether the operation was successful
- default: true
- concept_code:
- type: string
- title: Concept Code
- description: The concept code that was generated
- toml:
- type: string
- title: Toml
- description: Generated TOML content for the concept
- type: object
- required:
- - concept_code
- - toml
- title: BuildConceptResponse
- BuildInputsRequest:
- properties:
- files:
- anyOf:
- - items:
- $ref: '#/components/schemas/MthdsFileItem'
- type: array
- maxItems: 16
- minItems: 1
- - type: 'null'
- title: Files
- description: Inline MTHDS bundles forming the closure to resolve (content-passing — no server-side path reads).
- method_ref:
- anyOf:
- - type: string
- maxLength: 512
- minLength: 1
- - type: 'null'
- title: Method Ref
- description: Reference to a published method, resolving to its package's `.mthds` files. Address form (`github.com/<owner>/<repo>[/<selector>][@<tag>]`)
- is fetched and resolved server-side; the registry form stays reserved (501 until a method registry exists).
- pipe_ref:
- anyOf:
- - type: string
- maxLength: 256
- minLength: 1
- - type: 'null'
- title: Pipe Ref
- description: Qualified pipe ref (`domain.pipe_code`) to project. Optional — defaults to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`; a closure declaring none, or
- several, with no manifest `main_pipe` to settle it, requires it explicitly.
- format:
- $ref: '#/components/schemas/InputsTemplateFormat'
- description: Template encoding. `json` returns the parsed template in `inputs`; `toml` returns the raw text in `inputs_toml`.
- default: json
- explicit:
- type: boolean
- title: Explicit
- description: When true, emit the ceremonial `{concept, content}` envelope for every input. Defaults to false — the
- light, signature-driven shape that smart inputs accepts (a bare string for a Text input, and so on).
- default: false
- type: object
- title: BuildInputsRequest
- description: 'The inputs-template request: the shared closure + pipe selectors, plus the two rendering axes.
-
-
- Both axes mirror `pipelex codegen inputs` exactly — `--format` and `--explicit` — and every
-
- combination of them is served, as on the CLI.'
- BuildInputsValidReport:
- properties:
- is_valid:
- type: boolean
- const: true
- title: Is Valid
- default: true
- pipe_ref:
- type: string
- title: Pipe Ref
- description: The qualified pipe the template was generated for — the resolved selector.
- requested_pipe_ref:
- anyOf:
- - type: string
- - type: 'null'
- title: Requested Pipe Ref
- description: The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`.
- format:
- $ref: '#/components/schemas/InputsTemplateFormat'
- description: The template encoding (echo of the request).
- explicit:
- type: boolean
- title: Explicit
- description: Whether the ceremonial envelope shape was emitted (echo of the request).
- inputs:
- anyOf:
- - additionalProperties: true
- type: object
- - type: 'null'
- title: Inputs
- description: The parsed inputs template — present exactly when `format` is `json`.
- inputs_toml:
+ title: Title
+ concept_ref:
anyOf:
- type: string
- type: 'null'
- title: Inputs Toml
- description: The inputs template as TOML text — present exactly when `format` is `toml`.
- message:
- type: string
- title: Message
- description: Status message
- default: Inputs template generated successfully
- type: object
- required:
- - pipe_ref
- - format
- - explicit
- title: BuildInputsValidReport
- description: 'The 200 **valid** arm: the example inputs template for the requested pipe.
-
-
- The template rides **one of two fields, chosen by `format`** — `inputs` (parsed object) for
-
- `json`, `inputs_toml` (raw text) for `toml`. TOML cannot be carried as a parsed object without
-
- losing what makes it worth asking for (its concept comments and key order), and the JSON case
-
- must stay a real object, since that is what the deploy dialog and the SDKs consume. So the two
-
- are separate, honestly-typed fields, and the unused one is omitted from the body entirely.
-
-
- A pipe that declares no inputs is a *valid* verdict, not an error (the CLI likewise exits 0):
-
- the template is simply empty, and `message` says so.'
- BuildOutputRequest:
- properties:
- files:
+ title: Concept Ref
+ refines:
anyOf:
- items:
- $ref: '#/components/schemas/MthdsFileItem'
+ type: string
type: array
- maxItems: 16
- minItems: 1
- type: 'null'
- title: Files
- description: Inline MTHDS bundles forming the closure to resolve (content-passing — no server-side path reads).
- method_ref:
+ title: Refines
+ description:
anyOf:
- type: string
- maxLength: 512
- minLength: 1
- type: 'null'
- title: Method Ref
- description: Reference to a published method, resolving to its package's `.mthds` files. Address form (`github.com/<owner>/<repo>[/<selector>][@<tag>]`)
- is fetched and resolved server-side; the registry form stays reserved (501 until a method registry exists).
- pipe_ref:
+ title: Description
+ required:
+ type: boolean
+ title: Required
+ presence:
anyOf:
- - type: string
- maxLength: 256
- minLength: 1
+ - $ref: '#/components/schemas/PresenceMarker'
- type: 'null'
- title: Pipe Ref
- description: Qualified pipe ref (`domain.pipe_code`) to project. Optional — defaults to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`; a closure declaring none, or
- several, with no manifest `main_pipe` to settle it, requires it explicitly.
- format:
- $ref: '#/components/schemas/ConceptRepresentationFormat'
- description: Representation to render. `schema` (JSON Schema) and `json` (example value) return a parsed object
- in `output`; `python` returns Python source in `output_python`.
- default: schema
- type: object
- title: BuildOutputRequest
- description: 'The output-representation request: the shared closure + pipe selectors, plus the format axis.'
- BuildOutputValidReport:
- properties:
- is_valid:
- type: boolean
- const: true
- title: Is Valid
- default: true
- pipe_ref:
- type: string
- title: Pipe Ref
- description: The qualified pipe the representation was generated for — the resolved selector.
- requested_pipe_ref:
+ gating:
anyOf:
- - type: string
+ - type: boolean
- type: 'null'
- title: Requested Pipe Ref
- description: The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`.
- format:
- $ref: '#/components/schemas/ConceptRepresentationFormat'
- description: The representation format (echo of the request).
- output:
+ title: Gating
+ default_value:
+ title: Default Value
+ examples:
anyOf:
- - additionalProperties: true
- type: object
+ - items: {}
+ type: array
- type: 'null'
- title: Output
- description: The parsed output representation — present exactly when `format` is `schema` or `json`.
- output_python:
+ title: Examples
+ hints:
anyOf:
- - type: string
+ - additionalProperties:
+ type: string
+ type: object
- type: 'null'
- title: Output Python
- description: The output representation as Python source — present exactly when `format` is `python`.
- message:
- type: string
- title: Message
- description: Status message
- default: Output representation generated successfully
- type: object
- required:
- - pipe_ref
- - format
- title: BuildOutputValidReport
- description: 'The 200 **valid** arm: the example output representation for the requested pipe.
-
-
- The representation rides **one of two fields, chosen by `format`**, for the same reason
-
- `/build/inputs` splits `inputs` / `inputs_toml`: `schema` and `json` are objects, `python` is
-
- source text. (Before this split the route parsed *every* format as JSON, so `format=python` was a
-
- hard 500 — it fed Python source to `json.loads`.)'
- BuildPipeSpecRequest:
- properties:
- pipe_type:
- type: string
- maxLength: 128
- minLength: 1
- title: Pipe Type
- description: The pipe type (e.g. PipeLLM, PipeSequence, etc.).
- spec:
- additionalProperties: true
- type: object
- title: Spec
- description: JSON pipe specification.
- type: object
- required:
- - pipe_type
- - spec
- title: BuildPipeSpecRequest
- BuildPipeSpecResponse:
- properties:
- success:
- type: boolean
- title: Success
- description: Whether the operation was successful
- default: true
- pipe_code:
- type: string
- title: Pipe Code
- description: The pipe code that was generated
- pipe_type:
+ title: Hints
+ kind:
type: string
- title: Pipe Type
- description: The pipe type
- toml:
+ const: boolean
+ title: Kind
+ default: boolean
+ name:
type: string
- title: Toml
- description: Generated TOML content for the pipe
+ title: Name
+ additionalProperties: false
type: object
required:
- - pipe_code
- - pipe_type
- - toml
- title: BuildPipeSpecResponse
- BuildRunnerRequest:
+ - required
+ - name
+ title: BooleanField
+ description: '`boolean` at a named position.'
+ BooleanItem:
properties:
- analytics_groups:
+ title:
anyOf:
- - additionalProperties:
- type: string
- type: object
+ - type: string
- type: 'null'
- title: Analytics Groups
- description: 'PIPELEX-API EXTENSION (not part of the MTHDS Protocol) — the host-supplied groups the caller of this
- request belongs to, as a mapping of group type to group key (e.g. `{"organization": "org_acme"}`), exactly as
- a run request states them. The work this request does is not a run, but it still emits telemetry — the validation
- sweep''s `pipe_dry_run` event and its dry runs — and that telemetry is attributed to the caller: the authenticated
- user and these groups. Same rules and same 422 `InvalidAnalyticsGroups` refusal as on `/execute` and `/start`.
- Omit it and the caller belongs to no group.'
- files:
+ title: Title
+ concept_ref:
+ anyOf:
+ - type: string
+ - type: 'null'
+ title: Concept Ref
+ refines:
anyOf:
- items:
- $ref: '#/components/schemas/MthdsFileItem'
+ type: string
type: array
- maxItems: 16
- minItems: 1
- type: 'null'
- title: Files
- description: Inline MTHDS bundles forming the closure to resolve (content-passing — no server-side path reads).
- method_ref:
+ title: Refines
+ description:
anyOf:
- type: string
- maxLength: 512
- minLength: 1
- type: 'null'
- title: Method Ref
- description: Reference to a published method, resolving to its package's `.mthds` files. Address form (`github.com/<owner>/<repo>[/<selector>][@<tag>]`)
- is fetched and resolved server-side; the registry form stays reserved (501 until a method registry exists).
- pipe_ref:
+ title: Description
+ required:
+ type: boolean
+ title: Required
+ presence:
anyOf:
- - type: string
- maxLength: 256
- minLength: 1
+ - $ref: '#/components/schemas/PresenceMarker'
- type: 'null'
- title: Pipe Ref
- description: Qualified pipe ref (`domain.pipe_code`) to project. Optional — defaults to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`; a closure declaring none, or
- several, with no manifest `main_pipe` to settle it, requires it explicitly.
- allow_signatures:
- type: boolean
- title: Allow Signatures
- description: When true, the validation sweep tolerates unimplemented pipe signatures instead of rejecting the bundle
- (signatures dry-run trivially by minting a mock). Defaults to false (strict).
- default: false
- type: object
- title: BuildRunnerRequest
- description: 'The runner-script request: the shared closure + pipe selectors, plus the sweep''s `allow_signatures`.
-
-
- Alone among the `/build/*` projections this route keeps `allow_signatures`, because alone among
-
- them it still runs the dry-run sweep — the flag only ever parameterized that sweep.'
- BuildRunnerValidReport:
- properties:
- is_valid:
- type: boolean
- const: true
- title: Is Valid
- default: true
- pipe_ref:
- type: string
- title: Pipe Ref
- description: The qualified pipe the runner was generated for — the resolved selector.
- requested_pipe_ref:
+ gating:
anyOf:
- - type: string
+ - type: boolean
- type: 'null'
- title: Requested Pipe Ref
- description: The `pipe_ref` as submitted. Absent when it was omitted and defaulted — to the fetched package manifest's
- `main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`.
- python_code:
- type: string
- title: Python Code
- description: Generated Python script for running the pipeline, imports spelled with the emitted class names.
- structures:
- $ref: '#/components/schemas/RunnerStructures'
- description: The typed-structures projection the script imports from.
- message:
+ title: Gating
+ default_value:
+ title: Default Value
+ examples:
+ anyOf:
+ - items: {}
+ type: array
+ - type: 'null'
+ title: Examples
+ hints:
+ anyOf:
+ - additionalProperties:
+ type: string
+ type: object
+ - type: 'null'
+ title: Hints
+ kind:
type: string
- title: Message
- description: Status message
- default: Runner code generated successfully
+ const: boolean
+ title: Kind
+ default: boolean
+ additionalProperties: false
type: object
required:
- - pipe_ref
- - python_code
- - structures
- title: BuildRunnerValidReport
- description: 'The 200 **valid** arm: the runner script plus the structures projection it imports from.'
+ - required
+ title: BooleanItem
+ description: '`boolean` — true or false.'
CodegenRequest:
properties:
files:
@@ -2240,9 +1423,9 @@ components:
the promise this route''s valid arm makes by carrying a `lock`. Input templates are user-editable
- scaffolds, never stamped or locked, so they cannot make that promise; they ride
+ scaffolds, never stamped or locked, so they cannot make that promise: a client projects one from
- `POST /build/inputs` instead and `inputs` is therefore not a kind here. Future per-pipe kinds
+ the input form `POST /pipe-io` serves, and `inputs` is therefore not a kind here. Future per-pipe kinds
(`docs`, `tools`, `tests`) do emit tracked artifacts, so they join this enum and select their
@@ -2403,14 +1586,6 @@ components:
required:
- description
title: ConceptBlueprint
- ConceptRepresentationFormat:
- type: string
- enum:
- - json
- - python
- - schema
- title: ConceptRepresentationFormat
- description: Output format for concept representations.
ConceptStructureBlueprint:
properties:
description:
@@ -3761,13 +2936,6 @@ components:
`PipeBlueprint.collapse_hint_free_slot_tables`), so a slot value that IS a table carries
non-empty hints — hint-free bundles produce byte-identical blueprints to the string form.'
- InputsTemplateFormat:
- type: string
- enum:
- - json
- - toml
- title: InputsTemplateFormat
- description: Serialization format for a generated inputs template.
InvalidReport:
properties:
is_valid:
@@ -5462,9 +4630,9 @@ components:
minLength: 1
- type: 'null'
title: Pipe Ref
- description: Qualified pipe ref (`domain.pipe_code`) to project. Optional — defaults to the fetched package manifest's
+ description: Qualified pipe ref (`domain.pipe_code`) to select. Optional — defaults to the fetched package manifest's
`main_pipe` on a `method_ref` request, else to the closure's declared `main_pipe`; a closure declaring none, or
- several, with no manifest `main_pipe` to settle it, requires it explicitly.
+ several, with no manifest `main_pipe` to settle it, requires it unless `all_pipes` is set.
all_pipes:
type: boolean
title: All Pipes
@@ -6728,7 +5896,7 @@ components:
title: Validation Errors
description: Structured per-error diagnostics, carried by a `ValidateBundleError`. Each item may carry a `suggested_fix`.
Retained under `ERROR_DISCLOSURE=strict` — it describes the caller's own bundle, not server internals. On `/validate`,
- `/resolve`, `/codegen` and `/build/*` an invalid bundle is a **200** verdict instead, so the items ride the response
+ `/resolve`, `/codegen` and `/pipe-io` an invalid bundle is a **200** verdict instead, so the items ride the response
body there rather than a problem document.
additionalProperties: true
type: object
@@ -7149,40 +6317,6 @@ components:
- ERROR
title: RunState
description: Run lifecycle state — a pipelex extension field on run responses (the protocol defines none).
- RunnerStructures:
- properties:
- directory:
- type: string
- title: Directory
- description: Directory (relative to the runner script) to write the artifacts and lock into.
- default: structures
- artifacts:
- items:
- $ref: '#/components/schemas/GeneratedArtifact'
- type: array
- title: Artifacts
- description: The stamped generated files (paths relative to `directory`).
- lock:
- type: string
- title: Lock
- description: The lock content (TOML) tracking the artifact set — write verbatim inside `directory`.
- lock_filename:
- type: string
- title: Lock Filename
- description: Filename the lock content must be written as.
- default: codegen.lock
- type: object
- required:
- - lock
- title: RunnerStructures
- description: 'The typed-structures projection the runner script imports from (`from structures.structures import ...`).
-
-
- The same stamped `python-structures` artifacts + `codegen.lock` a local `pipelex build runner`
-
- scaffolds into `<output>/structures/` — write `artifacts` and the lock under `directory` and
-
- the returned `python_code` runs against them, with the offline `codegen check` passing there.'
SearchSetting:
properties:
model:
diff --git a/docs/api-server/pipe-builder.md b/docs/api-server/pipe-builder.md
index 99e5cfa13..50f44e414 100644
--- a/docs/api-server/pipe-builder.md
+++ b/docs/api-server/pipe-builder.md
@@ -1,184 +1,10 @@
-# Pipe Builder
+# Pipe Builder (`/v1/build/*`) — REMOVED
-Generate input templates, output representations, and Python runner code for a pipe of an MTHDS library closure.
+**The five build routes are gone from this server**: `POST /v1/build/inputs`, `POST /v1/build/output`, `POST /v1/build/runner`, `POST /v1/build/concept` and `POST /v1/build/pipe-spec` now answer `404`. Nothing in the runtime changed; only the HTTP routes were removed.
-All three build endpoints speak the same verdict discipline as [`POST /v1/validate`](pipe-validate.md): a **produced verdict is always a `200`** discriminated on the body's `is_valid` field. The valid arm carries the built artifact; the invalid arm carries the structured `validation_errors[]` built by pipelex's one shared error builder. Non-2xx is reserved for *no verdict could be produced* — request-shape `422`, auth `401`/`403`, server `5xx` — rendered as RFC 7807 `application/problem+json` (see [Error Responses](error-responses.md)).
+## What replaces each one
-## The shared request envelope
-
-All three take the same **closure selector** as [`POST /v1/resolve` and `POST /v1/codegen`](codegen.md) — inline `files[]` **XOR** a `method_ref` — plus a **pipe selector**:
-
-- `files` (list, required unless `method_ref`): the inline MTHDS bundles forming the closure. Each item is `{ "content": "<mthds text>", "source": "<optional logical path>" }`. The optional `source` is threaded onto the blueprint, so diagnostics point at the owning file.
-- `method_ref` (string): a reference to a published method. The **address form** — `github.com/<owner>/<repo>` plus an optional package selector and `@<tag>` — is resolved server-side through the same fetch path as a [`method_ref` run](pipe-run.md#running-a-method-by-address-method_ref): the package's `.mthds` files feed the closure (only `.mthds` data travels — the package's Python never loads here), and the package manifest's `main_pipe` becomes the default pipe selector. The **registry form** (any non-address reference) answers `501` (`MethodRefNotSupported`) until method-registry resolution lands.
-- `pipe_ref` (string, **optional**): the qualified pipe ref `domain.pipe_code` to project — the same selector as `pipelex codegen inputs --pipe`. Omitted, it defaults with the same precedence as a run by address: on a `method_ref` request, to the fetched package manifest's `main_pipe` (the package author's declared entry pipe); otherwise to the closure's own declared `main_pipe`. A closure that declares **no** `main_pipe`, or **several** across domains, cannot be defaulted from its declarations, so when no manifest `main_pipe` settles it an omitted `pipe_ref` is a `422` there. Inline `files[]` carry no manifest, so for them the default is always the closure's declaration. Every selection refusal is an input `422` whose `error_type` is the pipelex entry-lookup class that names the failure, not the `ValidationError` of a malformed request: `EntryPipeNotFoundError` for a `pipe_ref` that names no pipe, a manifest `main_pipe` the closure does not declare, or an omitted `pipe_ref` over a closure declaring no `main_pipe`; `EntryPipeAmbiguousError` for a bare code, whether the request or the manifest spelled it, that matches pipes in several domains, or an omitted `pipe_ref` over a closure declaring several `main_pipe`s. An ambiguity's `detail` names the qualified refs to choose from. An unknown or ambiguous `pipe_ref` answers the `error_type` the run routes answer for the same `pipe_code`; the two omitted-`pipe_ref` refusals follow the pipe-selector design, since a run over such a closure has no matching refusal today.
-
-Every valid arm echoes both `pipe_ref` (the ref actually projected) and `requested_pipe_ref` (the ref as submitted — **absent** when it was omitted and defaulted), so a caller can always see which pipe it got. The echoed `pipe_ref` is **always qualified**, read back off the resolved pipe: a bare code still resolves (the engine's lookup falls back across domains), but you are told `smoke.echo`, not the `echo` you sent.
-
-## Static projections vs. the runner
-
-`/build/inputs` and `/build/output` are **static**: they resolve the closure to its normalized crate and read the requested pipe's **declared** IO. There is no dry-run sweep, and therefore no `allow_signatures` flag (it only ever parameterized that sweep). A valid verdict from them says the closure is structurally sound and the projection matches what the pipe *declares* — it is **not** a promise the pipe runs. Runnability is [`/validate`](pipe-validate.md)'s vocabulary; ask it.
-
-`/build/runner` is the exception. A runner script *is* a promise the pipe can run, so it keeps the dry-run sweep and keeps `allow_signatures`.
-
-## Build Inputs
-
-Generate an example inputs template for a pipe — the same projection `pipelex codegen inputs` writes, on both of its axes.
-
-**Endpoint:** `POST /v1/build/inputs`
-
-**Request Body:**
-
-```json
-{
- "files": [
- { "content": "domain = \"cv_matching\"\nmain_pipe = \"analyze_cv_job_match\"\n\n[pipe.analyze_cv_job_match]\ntype = \"PipeLLM\"\ndescription = \"Analyze CV against job offer\"\ninputs = { cv_text = \"native.Text\", job_offer = \"native.Text\" }\noutput = \"MatchAnalysis\"", "source": "cv_matching.mthds" }
- ],
- "pipe_ref": "cv_matching.analyze_cv_job_match"
-}
-```
-
-**Request Fields:** the shared envelope above, plus:
-
-- `format` (string, optional): `"json"` (default) or `"toml"`.
-- `explicit` (boolean, optional, default `false`): when `true`, emit the ceremonial `{concept, content}` envelope for every input. The default is the *light*, signature-driven shape — a bare string for a Text-refining input, a bare number for a Number-refining one, and so on — which is what smart inputs accepts.
-
-**The template rides the field its `format` names.** JSON is a parsed object in `inputs`; TOML is raw text in `inputs_toml`. TOML cannot ride as a parsed object without losing the concept comments and key order that are the reason to ask for it, and the JSON case must stay a real object because that is what clients feed straight into a run. The field the format did not select is **omitted from the body entirely**.
-
-**Response (valid verdict, default light JSON):**
-
-```json
-{
- "is_valid": true,
- "pipe_ref": "cv_matching.analyze_cv_job_match",
- "requested_pipe_ref": "cv_matching.analyze_cv_job_match",
- "format": "json",
- "explicit": false,
- "inputs": {
- "cv_text": "text_value",
- "job_offer": "text_value"
- },
- "message": "Inputs template generated successfully"
-}
-```
-
-With `"explicit": true`, each value becomes its `{ "concept": "native.Text", "content": { "text": "text_value" } }` envelope instead. With `"format": "toml"`, the body carries `inputs_toml` (a string) in place of `inputs`.
-
-A pipe that declares **no inputs** is a *valid* verdict, not an error: the template is simply empty (`{}` / `""`) and `message` says so — mirroring the CLI, which exits 0 on it.
-
-**Response (invalid verdict):** `200` with `is_valid: false` and `validation_errors[]` — the same invalid arm every build endpoint returns:
-
-```json
-{
- "is_valid": false,
- "validation_errors": [
- { "category": "blueprint_validation", "message": "...", "source": "cv_matching.mthds" }
- ],
- "message": "..."
-}
-```
-
-An item may also carry a `suggested_fix` — a structured, deterministic repair the runtime derived for that error. See [Error Responses → Suggested fixes](error-responses.md#suggested-fixes).
-
----
-
-## Build Output
-
-Generate the output representation for a pipe in one of three formats.
-
-**Endpoint:** `POST /v1/build/output`
-
-**Request Body:**
-
-```json
-{
- "files": [{ "content": "...your MTHDS content..." }],
- "pipe_ref": "cv_matching.analyze_cv_job_match",
- "format": "schema"
-}
-```
-
-**Request Fields:** the shared envelope above, plus:
-
-- `format` (string, optional): `"schema"` (default), `"json"`, or `"python"`. **Must be lowercase.**
-
-**Format Options:**
-
-- `schema` — JSON Schema representation of the output concept → parsed object in `output`
-- `json` — Example JSON instance of the output → parsed object in `output`
-- `python` — Python source defining the output class → raw text in `output_python`
-
-As with `/build/inputs`, the representation rides the field its `format` names, and the other is omitted.
-
-**Response (valid verdict, schema format):**
-
-```json
-{
- "is_valid": true,
- "pipe_ref": "cv_matching.analyze_cv_job_match",
- "requested_pipe_ref": "cv_matching.analyze_cv_job_match",
- "format": "schema",
- "output": {
- "concept": "MatchAnalysis",
- "content": {
- "type": "object",
- "properties": {
- "score": { "title": "Score", "type": "integer", "description": "Match score 0-100" }
- },
- "required": ["score"]
- }
- },
- "message": "Output representation generated successfully"
-}
-```
-
-One no-verdict special case: a pipe whose output is `native.Anything` and whose concrete options cannot be determined has no representation to render — a request-shape `422`, since that is a fact about the requested pipe, not about the closure.
-
-The invalid verdict is the shared `is_valid: false` arm shown under Build Inputs.
-
----
-
-## Build Runner
-
-Generate a Python runner script for executing a pipe. The script's imports, example inputs, and output cast are spelled with the **emitted** class names of the typed-structures projection, and the response carries that projection alongside the script — the same stamped `structures.py` + `codegen.lock` a local `pipelex build runner` scaffolds (see [Resolve & Codegen](codegen.md) for the stamp/lock trust chain).
-
-**Endpoint:** `POST /v1/build/runner`
-
-**Request Body:**
-
-```json
-{
- "files": [{ "content": "...your MTHDS content..." }],
- "pipe_ref": "cv_matching.analyze_cv_job_match"
-}
-```
-
-**Request Fields:** the shared envelope above, plus:
-
-- `allow_signatures` (boolean, optional): tolerate unimplemented pipe signatures in the dry-run sweep (default `false`). This is the only build route that takes it — the only one that still sweeps.
-- `analytics_groups` (object | null, optional): the groups the caller belongs to, with the same rules and the same 422 `InvalidAnalyticsGroups` as [on `/validate`](pipe-validate.md#who-the-validation-is-done-for). The sweep is done for the authenticated user and these groups, so its telemetry is attributed to them.
-
-The sweep is scoped to the requested pipe, so unrelated broken siblings do not block a good pipe. When `pipe_ref` is omitted the scope is not settled before the closure loads, so the **whole closure** is swept and the pipe then defaults as described in the shared envelope above — the fetched manifest's `main_pipe` on a `method_ref` request, else the closure's own — a stricter verdict, and the honest one for a caller who did not say which pipe they meant.
-
-**Response (valid verdict):**
-
-```json
-{
- "is_valid": true,
- "pipe_ref": "cv_matching.analyze_cv_job_match",
- "requested_pipe_ref": "cv_matching.analyze_cv_job_match",
- "python_code": "import sys\nfrom pathlib import Path\n...",
- "structures": {
- "directory": "structures",
- "artifacts": [
- { "path": "structures.py", "content": "# >>> pipelex-codegen-stamp >>>\n..." }
- ],
- "lock": "# codegen.lock — generated artifact set (Pipelex codegen). Do not edit by hand.\n...",
- "lock_filename": "codegen.lock"
- },
- "message": "Runner code generated successfully"
-}
-```
-
-To materialize a runnable tree, write `python_code` as the runner script and each `structures.artifacts[]` entry (plus `structures.lock` as `structures.lock_filename`) into the `structures.directory` beside it — the script imports from there (`from structures.structures import ...`).
-
-The invalid verdict — including a failed dry-run of the requested pipe — is the shared `is_valid: false` arm shown under Build Inputs. One no-verdict special case: a requested pipe whose cross-package dependencies are absent from the request (recorded SKIPPED by the sweep) is a request-shape `422`, since no runner can be honestly generated without its dependency closure.
+- **An inputs template (`/v1/build/inputs`)** is projected by the client from the input form that [`POST /v1/pipe-io`](pipe-io.md) returns, with the `mthds` packages' projection: `renderInputsTemplate` from `mthds/protocol`, which `@pipelex/sdk` re-exports, or `mthds.protocol.inputs_template` in Python. `mthds-agent inputs` does exactly that. `pipelex build inputs` still writes the engine's own rendering locally, the template the removed route returned, which differs from the client projection where the [projection fixture corpus](../contribute/generate-projection-corpus.md) records.
+- **A pipe's declared output (`/v1/build/output`)** is described by the `output_form` and the pipe I/O contract that [`POST /v1/pipe-io`](pipe-io.md) returns. The rendered output example has no HTTP replacement; `pipelex build output` renders it locally.
+- **The typed structures a runner script imported (`/v1/build/runner`)** are the `python-structures` target of [`POST /v1/codegen`](codegen.md). The runner script itself has no HTTP replacement.
+- **The concept and pipe spec conversions (`/v1/build/concept`, `/v1/build/pipe-spec`)** have no replacement: a method is written as MTHDS directly and checked with [`POST /v1/validate`](pipe-validate.md).
diff --git a/docs/api-server/pipe-io.md b/docs/api-server/pipe-io.md
index 6fd4ab6bf..9949c6f2a 100644
--- a/docs/api-server/pipe-io.md
+++ b/docs/api-server/pipe-io.md
@@ -4,7 +4,7 @@ Return a method's I/O artifacts — its pipe I/O contracts, its input form and i
**Endpoint:** `POST /v1/pipe-io`
-The route resolves the closure through the same static core as [`POST /v1/resolve`](codegen.md#resolve), selects a pipe the way the [per-pipe build routes](pipe-builder.md) do, and derives the three artifacts with the builder that [`POST /v1/validate`](pipe-validate.md) and every run already call. It runs **no dry-run sweep**, so a call costs one load and one derivation, where `/v1/validate` mock-runs every pipe of the method. A caller that shows a method, prepares its inputs or generates types for it reads this route; a caller that needs the dry-run verdict or the dry-run graph stays on `/v1/validate`, whose `views` still attach both forms beside the verdict.
+The route resolves the closure through the same static core as [`POST /v1/resolve`](codegen.md#resolve), selects a pipe with the [selection chain](#pipe-selection) below, and derives the three artifacts with the builder that [`POST /v1/validate`](pipe-validate.md) and every run already call. It runs **no dry-run sweep**, so a call costs one load and one derivation, where `/v1/validate` mock-runs every pipe of the method. A caller that shows a method, prepares its inputs or generates types for it reads this route; a caller that needs the dry-run verdict or the dry-run graph stays on `/v1/validate`, whose `views` still attach both forms beside the verdict.
It is a **Pipelex API extension**, not an MTHDS Protocol route: it is not tagged `x-mthds-protocol` in the [OpenAPI artifact](openapi/pipelex-api.openapi.yaml). The artifacts it carries are the standard's, under their neutral names — see the MTHDS specification's pages on [pipe I/O contracts, the input-form descriptor and the output-form descriptor](https://mthds.ai).
@@ -31,7 +31,7 @@ There is no `views` field, because the valid arm always carries all three artifa
## Pipe selection
-The selection chain is the one the build routes share: the request's `pipe_ref`; else a fetched package's manifest `main_pipe`; else the closure's own `main_pipe` declaration, when exactly one domain declares one. The chain stops at the first link that is present, so a manifest `main_pipe` the closure does not declare, or declares in several domains, is a failed selection rather than a fall-through to the closure's declarations.
+The selection chain takes the request's `pipe_ref`; else a fetched package's manifest `main_pipe`; else the closure's own `main_pipe` declaration, when exactly one domain declares one. The chain stops at the first link that is present, so a manifest `main_pipe` the closure does not declare, or declares in several domains, is a failed selection rather than a fall-through to the closure's declarations.
The closure is resolved first, so an invalid closure answers its invalid verdict whatever `pipe_ref` names. For a valid closure these selections are refused with an input `422` `problem+json`. Its `error_type` is the pipelex entry-lookup class that names the failure, never the `ValidationError` of a malformed request, so a client tells a selection refusal from a request-shape one by that field:
diff --git a/docs/api-server/pipe-validate.md b/docs/api-server/pipe-validate.md
index 5570e7f65..3bc76f962 100644
--- a/docs/api-server/pipe-validate.md
+++ b/docs/api-server/pipe-validate.md
@@ -126,7 +126,7 @@ The field exists because the canonical report is **manifest-blind**: `bundle_blu
The graph follows the same precedence: on a `method_ref` request whose manifest names a `main_pipe`, the route hands that pipe to the runtime as the graph target, so `graph_spec` draws the pipe `default_pipe_ref` names — including for a package whose bundles declare no `main_pipe` at all, whose entry pipe only the manifest states. A manifest `main_pipe` the closure does not resolve leaves both fields `null` rather than graphing the closure's own `main_pipe`, which no run by that address executes. The target travels with the dispatch, so a validation sent to a worker is graphed the same way.
-It states the **run** default, which is looser than the `/build/*` routes' rule on one point: a closure whose domains each declare a `main_pipe` cannot be defaulted on `/build/*` (a `422`), but `/execute` and `/start` run its first declaring blueprint happily — so this field names that pipe rather than reporting `null`.
+It states the **run** default, which is looser than the [`/pipe-io`](pipe-io.md#pipe-selection) selection chain on one point: a closure whose domains each declare a `main_pipe` cannot be defaulted on `/pipe-io` (a `422`), but `/execute` and `/start` run its first declaring blueprint happily — so this field names that pipe rather than reporting `null`.
The field rides the valid arm only. The invalid arm assembles no library, so there is no entry pipe to name and the field is absent, like the other structural artifacts.
diff --git a/docs/contribute/generate-projection-corpus.md b/docs/contribute/generate-projection-corpus.md
index eacca2a37..b5f0c7ab8 100644
--- a/docs/contribute/generate-projection-corpus.md
+++ b/docs/contribute/generate-projection-corpus.md
@@ -1,6 +1,6 @@
# Generating the Projection Fixture Corpus
-The MTHDS build routes no longer render a pipe's fill-in inputs template on the server. Each client SDK projects it from the input-form descriptor it already has — once in TypeScript (`mthds`) and once in Python (`mthds-python`) — and the two projections must agree **exactly**, TOML comment lines included, or the JS/Python asymmetry the change set out to remove is simply rebuilt one layer up.
+No server route renders a pipe's fill-in inputs template. Each client SDK projects it from the input-form descriptor it already has — once in TypeScript (`mthds`) and once in Python (`mthds-python`) — and the two projections must agree **exactly**, TOML comment lines included, or the JS/Python asymmetry the change set out to remove is simply rebuilt one layer up.
`pipelex-dev generate-projection-corpus` writes the fixture corpus that makes that agreement checkable. It is the **sole producer** of the capture committed byte-identically in `mthds-js/tests/fixtures/protocol/` and `mthds-python/tests/fixtures/protocol/`; `trace-input-semantics` stays what its own page says it is, a debugging tracer.
diff --git a/docs/under-the-hood/codegen-projections.md b/docs/under-the-hood/codegen-projections.md
index a302aa50f..69fe68caa 100644
--- a/docs/under-the-hood/codegen-projections.md
+++ b/docs/under-the-hood/codegen-projections.md
@@ -167,7 +167,7 @@ The remaining findings are artifacts of *how* you lint, not of file content. Non
## Serving the engine over HTTP
-The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen`, and the re-pointed `/v1/build/*` — the route envelopes are documented on the API server's [Resolve & Codegen](../api-server/codegen.md) page). Two host-facing cores make that possible without any CLI plumbing:
+The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen` and `POST /v1/pipe-io` — the route envelopes are documented on the API server's [Resolve & Codegen](../api-server/codegen.md) and [Pipe I/O](../api-server/pipe-io.md) pages). Two host-facing cores make that possible without any CLI plumbing:
- **`pipelex.pipeline.resolve_bundle.resolve_crate_from_contents`** resolves **in-memory** MTHDS contents (strings, with optional per-content sources) into the normalized crate. It mirrors `validate_bundle`'s in-memory arm — the same `translate_to_validate_bundle_error` cascade, so an invalid library raises the one shared `ValidateBundleError` and a resolve verdict cannot drift from a validate verdict — and the same **loaded-on-success contract**: the library is left loaded and current for the host to read live pipes from, and the host owns its teardown. Resolution is static (no dry-run sweep), matching `pipelex resolve`.
- **`build_stamped_projection`** (above) gives the host the stamped artifact set plus the lock as pure content. A client that writes the served files and lock verbatim reproduces a local run byte-for-byte — the offline `codegen check` passes on the written tree exactly as it would locally. There is deliberately no server-side check route: the check is offline by design.
diff --git a/docs/under-the-hood/input-form-descriptor.md b/docs/under-the-hood/input-form-descriptor.md
index c1387c458..7bd0bcdfd 100644
--- a/docs/under-the-hood/input-form-descriptor.md
+++ b/docs/under-the-hood/input-form-descriptor.md
@@ -77,4 +77,4 @@ Inapplicable slots are absent, never JSON `null`: the report's valid arm is dump
## What clients project from it
-The descriptor is what a client SDK renders a pipe's fill-in inputs template from — the hosted build routes no longer render one server-side. That projection is written twice, in TypeScript and in Python, and the two are pinned against each other by a shared fixture corpus this engine generates: see [Projection Fixture Corpus](../contribute/generate-projection-corpus.md). The corpus is also where the projection's rules are stated, including the places it is deliberately right where this engine's own inputs-template renderer is not.
+The descriptor is what a client SDK renders a pipe's fill-in inputs template from — no server route renders one. That projection is written twice, in TypeScript and in Python, and the two are pinned against each other by a shared fixture corpus this engine generates: see [Projection Fixture Corpus](../contribute/generate-projection-corpus.md). The corpus is also where the projection's rules are stated, including the places it is deliberately right where this engine's own inputs-template renderer is not.
diff --git a/mkdocs.yml b/mkdocs.yml
index 217883bc3..aef523f06 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -386,7 +386,6 @@ plugins:
- api-server/pipe-validate.md: "API Server Pipe Validate"
- api-server/pipe-io.md: "API Server Pipe I/O"
- api-server/codegen.md: "API Server Resolve & Codegen"
- - api-server/pipe-builder.md: "API Server Pipe Builder"
- api-server/mthds-tools.md: "API Server MTHDS Tools"
- api-server/error-responses.md: "API Server Error Responses"
- api-server/logging.md: "API Server Logging"
@@ -643,9 +642,9 @@ nav:
- Pipe Validate: api-server/pipe-validate.md
- Pipe I/O: api-server/pipe-io.md
- Resolve & Codegen: api-server/codegen.md
- - Pipe Builder: api-server/pipe-builder.md
- MTHDS Tools: api-server/mthds-tools.md
- Error Responses: api-server/error-responses.md
- Logging: api-server/logging.md
- Storage Transport (Removed): api-server/storage-transport.md
+ - Pipe Builder (Removed): api-server/pipe-builder.md
- Changelog to v0.33.2: api-server/changelog.md