Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

feature/Postman-skill-sends-files · L-261020-8f64a3 (#1463)

Louis Choquel5773985aadcb
diff --git a/api/.claude/commands/update-postman.md b/api/.claude/commands/update-postman.md
index 520bfe32e..1332be708 100644
--- a/api/.claude/commands/update-postman.md
+++ b/api/.claude/commands/update-postman.md
@@ -14,10 +14,10 @@ You update the **Pipelex FastAPI** collection directly in the user's Postman des
## Prerequisites
-You need the `POSTMAN_API_KEY` environment variable. **Always** run `source ~/.zshrc 2>/dev/null` before any command that uses it.
+You need the `POSTMAN_API_KEY` environment variable. **Always** run `source ~/.zshenv 2>/dev/null` before any command that uses it.
If `POSTMAN_API_KEY` is not set after sourcing, tell the user:
-> I need your Postman API key. Generate one in Postman: **Settings > API keys > Generate API Key**. Then add to `~/.zshrc`:
+> I need your Postman API key. Generate one in Postman: **Settings > API keys > Generate API Key**. Then add to `~/.zshenv`:
> ```
> export POSTMAN_API_KEY="your-key-here"
> ```
@@ -27,7 +27,7 @@ If `POSTMAN_API_KEY` is not set after sourcing, tell the user:
### 1. Fetch the current collection from Postman
```bash
-source ~/.zshrc 2>/dev/null
+source ~/.zshenv 2>/dev/null
curl -s -H "X-API-Key: $POSTMAN_API_KEY" \
"https://api.getpostman.com/collections/35082494-559c5753-885c-409a-af63-7647fe28d301" \
> /tmp/postman_current.json
@@ -130,8 +130,9 @@ For **unchanged endpoints**, do NOT touch them.
- All examples must be **self-contained** and **copy-pasteable**
- Response bodies must match **actual API responses** (from step 6)
- Read `api/postman/sample-bundles/` for `.mthds` files and their `.inputs.json` companions
+- Send each method as `files`, a map from each file's tree path to its content: for a sample bundle, `{"<name>.mthds": "<its text>"}`. Build the body of a method that has a directory of its own with the `postman-bundle` skill's script in preview mode (`--dry-run`), which lists every file of the method's tree by the standard's membership rule
-#### Pipe type diversity — for endpoints accepting `mthds_contents`:
+#### Pipe type diversity — for endpoints accepting a method's `files`:
1. **PipeLLM** — text analysis, summarization, Q&A
2. **PipeExtract** — structured data extraction
3. **PipeSequence** — multi-step pipeline
@@ -236,7 +237,7 @@ Key patterns:
### 9. Push to Postman
```bash
-source ~/.zshrc 2>/dev/null
+source ~/.zshenv 2>/dev/null
curl -s -X PUT \
-H "X-API-Key: $POSTMAN_API_KEY" \
-H "Content-Type: application/json" \
@@ -255,7 +256,7 @@ rm -f /tmp/postman_current.json /tmp/postman_updated.json
### 10. Verify
```bash
-source ~/.zshrc 2>/dev/null
+source ~/.zshenv 2>/dev/null
curl -s -H "X-API-Key: $POSTMAN_API_KEY" \
"https://api.getpostman.com/collections/35082494-559c5753-885c-409a-af63-7647fe28d301" \
| python3 -c "
@@ -295,7 +296,7 @@ Changes appear in Postman desktop immediately (auto-sync).
- **Do NOT write JSON files to the repo** — only use `/tmp/` for the API push, then clean up
- **Preserve user edits** — never overwrite unchanged endpoints
- **Show the change plan before pushing** — never push silently
-- **Always `source ~/.zshrc 2>/dev/null`** before any command using `$POSTMAN_API_KEY`
+- **Always `source ~/.zshenv 2>/dev/null`** before any command using `$POSTMAN_API_KEY`
- The Postman API expects `{"collection": {...}}` envelope — the fetched JSON already has this structure, so just modify `data['collection']` and write `data` back
- Enum values are lowercase: `schema`, `json`, `python` — NOT uppercase
-- `mthds_contents` is always `list[str]`, even for a single file
+- A method travels as `files`, a JSON object mapping each file's tree path to its UTF-8 text, or to `{"base64": "..."}` for a file whose bytes are not UTF-8, even for a method of one file. Never write the by-hash entry `{"sha256": ..., "size": ...}` into a collection request: it resolves only once the bytes are uploaded to the runner (`POST /v1/blobs/missing`, then `PUT /v1/blobs/{sha256}`), which a saved request never does. Never send `mthds_contents`, `mthds_sources`, `bundle_b64` or `resources`
diff --git a/api/.claude/skills/postman-bundle/SKILL.md b/api/.claude/skills/postman-bundle/SKILL.md
index dabbbad54..9ea3ac659 100644
--- a/api/.claude/skills/postman-bundle/SKILL.md
+++ b/api/.claude/skills/postman-bundle/SKILL.md
@@ -13,7 +13,8 @@ description: >-
API", "test /v1/resolve on this bundle", "give me the curl
for this bundle", or just "postman" said alongside a bundle path. It resolves the bundle exactly like
`pipelex run bundle <path>` (auto-detects bundle.mthds / the single .mthds, reads main_pipe, loads the
- sibling inputs.json). Trigger it even when the user does not name the endpoint or the word "skill".
+ sibling inputs.json) and sends the method's whole tree as `files`. Trigger it even when the user does not
+ name the endpoint or the word "skill".
---
# Postman Bundle
@@ -24,13 +25,13 @@ This is a focused, one-bundle-at-a-time tool. It is **not** the full collection
## Endpoint cheat sheet
-| `--endpoint` | Route | Body envelope | Needs a pipe? | Needs `inputs`? | Inference? |
+| `--endpoint` | Route | Body beside `files` | Needs a pipe? | Needs `inputs`? | Inference? |
|---|---|---|---|---|---|
-| `execute` | `POST /v1/execute` | run (`pipe_code`) | yes | yes | **YES — costs money** |
-| `start` | `POST /v1/start` | run + `callback_urls` | yes | yes | **YES — costs money** |
-| `validate` | `POST /v1/validate` | `mthds_contents` | no | no | no (free dry-run) |
-| `resolve` | `POST /v1/resolve` | `files[]` crate | no | no | no (free) |
-| `codegen` | `POST /v1/codegen` | `files[]` crate + `kind`/`target` | no | no | no (free) |
+| `execute` | `POST /v1/execute` | `pipe_code`, `inputs` | yes | yes | **YES — costs money** |
+| `start` | `POST /v1/start` | `pipe_code`, `inputs`, `callback_urls` | yes | yes | **YES — costs money** |
+| `validate` | `POST /v1/validate` | `allow_signatures`, optional `render` | no | no | no (free dry-run) |
+| `resolve` | `POST /v1/resolve` | nothing | no | no | no (free) |
+| `codegen` | `POST /v1/codegen` | `kind`, `target` | no | no | no (free) |
Every non-run endpoint answers with the **`/validate` verdict discipline**: a produced verdict is always a `200` discriminated on `is_valid` — the artifact on the valid arm, structured `errors[]` problem items on the invalid arm. Non-2xx is reserved for no-verdict conditions (request-shape `422`, auth `401`, server `5xx`), rendered as RFC 9457 `problem+json` with a registry `code`.
@@ -49,32 +50,35 @@ Which requests appear depends on `--endpoint`: the default `both` is `execute` +
(The top folder used to be `Run Bundle/` — if a stale folder by that name lingers in the collection, it can be deleted.)
-## The two request envelopes
+## The request body: the method as `files`
-**Run + validate routes** carry the bundle as parallel lists — `mthds_contents` (full text of each `.mthds` file, main one first) plus, on `/validate`, `mthds_sources` (one filename per content, so `errors[].location.file` names the owning file):
+Every route carries the method the same way: `files`, the MTHDS standard's map of file entries. Each key is a file's tree path, its path from the method tree's root, and each value is the file's UTF-8 text, or `{"base64": "..."}` for a file whose bytes are not UTF-8, such as an image or an office template:
```json
{
- "pipe_code": "<the bundle's main_pipe>",
- "mthds_contents": ["<file text>", "..."],
+ "pipe_code": "greeting.greet",
"inputs": { "...": "inputs.json copied verbatim" },
- "allow_signatures": false
+ "files": {
+ "bundle.mthds": "domain = \"greeting\"\nmain_pipe = \"greet\"\n...",
+ "templates/card.html": "<p>Hello, {{ text }}</p>\n",
+ "assets/logo.png": { "base64": "iVBORw0KGgo..." }
+ }
}
```
-(`pipe_code` and `inputs` ride execute/start only; `allow_signatures` rides validate.)
+`pipe_code` and `inputs` ride execute/start only, `allow_signatures` and the optional `render` ride validate, and `kind`/`target` ride codegen; `resolve` sends `files` alone. The **start** body additionally carries `callback_urls` (see [Callback URLs](#callback-urls-the-async-start-endpoint)). `files` comes last in every body, so the fields you edit in Postman stay on top.
-**Crate routes** (`resolve`, `codegen`) pair each content with its source in one `files[]` entry — the envelope the codegen spec pins. No `inputs`; the closure is the whole request:
+What travels is **every file of the method's tree**, listed by the `mthds` package's own lister, so it is judged by the same membership rule the runner and every MTHDS client apply (the standard's `method-tree.md`):
-```json
-{
- "kind": "types",
- "target": "ts-zod",
- "files": [{ "content": "<file text>", "source": "<file name>" }]
-}
-```
+- **The root** is the nearest directory at or above the bundle holding `METHODS.toml`, else the directory you point at. A `.mthds` file named alone with no `METHODS.toml` above it travels as the one-file tree of that file, nothing beside it.
+- **What stays out**: every path segment starting with a dot (`.git/`, `.venv/`, `.env`, `.mthds/`), `__pycache__` and `node_modules`, virtual environments, nested repositories and nested packages, the tree's own `.gitignore`d files and the manifest's `[package] exclude` patterns.
+- **A run's inputs file never travels as a file.** The standard keeps every file named `inputs.json` or `inputs.toml`, at any depth, out of every tree, and a runner applying that rule refuses such a key. The `mthds` lister pinned here still lists it, so the script drops it from `files` itself: the sibling `inputs.json` travels as `inputs` on execute/start and nowhere else.
+- **Never by hash.** The standard's third entry form, `{"sha256", "size"}`, names bytes the runner already holds in its blob store, which only an upload (`POST /v1/blobs/missing`, then `PUT /v1/blobs/{sha256}`) puts there. A Postman query or a curl command never uploads, so every file travels inline.
+- **No retired fields.** The skill never sends `mthds_contents`, `mthds_sources`, `bundle_b64` or `resources`.
+
+The listing stops at the runner's own limits on a tree a request sends (the `MAX_METHOD_TREE_*` limits of `pipelex_api.limits`), so pointing the skill at a large directory by mistake fails in moments. The script also refuses a main bundle the membership rule keeps out of the tree, such as a `.gitignore`d one, since the `pipe_code` read from it would name a pipe the runner never receives.
-(`kind`/`target` are codegen-only.) The **start** body additionally carries `callback_urls` (see [Callback URLs](#callback-urls-the-async-start-endpoint)). The **validate** body optionally carries `render` (see below).
+The `pipe_code` the script infers is the main bundle's `main_pipe` **qualified by that bundle's domain** (`greeting.greet`, not `greet`): the whole tree travels, so a sibling bundle declaring a pipe of the same code would make a bare code ambiguous, which the runner refuses. An explicit `--pipe` is sent exactly as given.
## Validate = the API's dry-run
@@ -96,7 +100,7 @@ Which requests appear depends on `--endpoint`: the default `both` is `execute` +
The valid arm carries the **stamped** artifact set plus its `codegen.lock`: a client that writes them verbatim reproduces a local `pipelex codegen types` run **byte-for-byte** and passes the offline `pipelex codegen check`. There is deliberately no server-side check route (the drift check is offline by design). An unknown `kind`/`target`, or a `pipe_ref` on the concept-set-wide `types`, is a request-shape `422`.
-`--target` is **required** for codegen — there is no default; ask the user which consumer they're generating for if unclear. The request envelope also accepts a `method_ref` instead of `files[]` (an address-form reference is fetched and resolved; only the registry form answers `501`) — this skill always sends `files[]`.
+`--target` is **required** for codegen — there is no default; ask the user which consumer they're generating for if unclear. The request also accepts a `method_ref` instead of `files` (an address-form reference is fetched and resolved; the registry form is a `422 request.unsupported`) — this skill always sends `files`.
## Callback URLs (the async `start` endpoint)
@@ -114,7 +118,7 @@ This only affects `start` — no other endpoint takes `callback_urls`. The API's
## How to run it
-Every command on this page runs from `api/`, the server's directory in the pipelex repository: `cd api` first, or prefix the make targets with `-C api` from the repository root (`make -C api bundle-run …`), and give `BUNDLE` and the script's paths relative to `api/`. The make targets are the convenient wrapper (they run the script with the server's venv):
+Every command on this page runs from `api/`, the server's directory in the pipelex repository: `cd api` first, or prefix the make targets with `-C api` from the repository root (`make -C api bundle-run …`), and give `BUNDLE` and the script's paths relative to `api/`. The script lists the method's tree with the `mthds` package and reads the runner's limits from `pipelex_api`, so it runs with the server's own environment, `api/.venv`, never a bare `python3`. The make targets are the convenient wrapper: they install that environment, then run the script with it.
```bash
make bundle-run BUNDLE=<dir|.mthds> # POST to a running API, print the response
@@ -131,13 +135,13 @@ make bundle-dry BUNDLE=<dir|.mthds> # print the request bod
`bundle-validate` / `bundle-resolve` / `bundle-codegen` hardcode `--run` with their endpoint (they're free and safe to run live).
-Under the hood it's one script that resolves the bundle once and sends the request to one of four sinks. `<bundle-path>` is a bundle **directory** (preferred) or a single `.mthds` file. Call the script directly for a path outside the repo or finer control:
+Under the hood it's one script that resolves the bundle once and sends the request to one of four sinks. `<bundle-path>` is a bundle **directory** (preferred) or a single `.mthds` file. Call the script directly for a path outside the repo or finer control, with the server's environment (`make install` creates it when it is missing):
**Push into the Postman collection** (default). Source the env first so the Postman key is present (it lives in `~/.zshenv`):
```bash
source ~/.zshenv 2>/dev/null
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> [--endpoint <ep>] [--target <t>]
+.venv/bin/python .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> [--endpoint <ep>] [--target <t>]
```
After a successful push, tell the user it's in Postman (it auto-syncs), and that they should set `base_url` and `auth_token` in their Postman environment, then Send.
@@ -145,7 +149,7 @@ After a successful push, tell the user it's in Postman (it auto-syncs), and that
**Run it yourself from Claude Code** — POSTs the exact same body to a running API and prints the response:
```bash
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint <ep>
+.venv/bin/python .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint <ep>
# defaults to http://127.0.0.1:8081 and no auth (AUTH_MODE=none). Override:
# --base-url https://api.pipelex.com --token <bearer>
```
@@ -153,12 +157,12 @@ python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-pat
The API must be up — start it locally with `make run` first. **Heads-up: only `execute`/`start` trigger real inference (cost + latency)** — confirm the user wants a live run for those. Every other endpoint is free and safe to fire without confirmation:
```bash
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint validate
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint resolve
-python3 .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint codegen --target python-pydantic
+.venv/bin/python .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint validate
+.venv/bin/python .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint resolve
+.venv/bin/python .claude/skills/postman-bundle/scripts/build_postman_query.py <bundle-path> --run --endpoint codegen --target python-pydantic
```
-**Emit a curl command** (`--curl`) — writes the body to a temp file and prints a `curl --data @file`, so the multi-line contents never need shell escaping. **Preview only** (`--dry-run`) — print the request body, touch nothing.
+**Emit a curl command** (`--curl`) — writes the body to a temp file and prints a `curl --data @file`, so the files' contents never need shell escaping. **Preview only** (`--dry-run`) — print the request body, touch nothing. Every mode prints the tree's root and the files it sends first, marking those that travel in Base64.
Useful options (apply across modes):
@@ -169,7 +173,7 @@ Useful options (apply across modes):
| `--allow-signatures` | (validate only) Tolerate unimplemented pipe signatures. Default strict. |
| `--render <fmt>` | (validate only) Opt-in server-side view, e.g. `--render markdown` → response carries `rendered_markdown` (what a skill-driven validate sends). Omit for the lean structured body a hook reads. Repeatable. |
| `--inputs <path>` | Inputs JSON lives elsewhere, or you point at a `.mthds` file directly (file mode does not auto-detect inputs). execute/start only. |
-| `--pipe <pipe_code>` | The bundle has no `main_pipe`, or you want a different pipe. Used by execute/start (as `pipe_code`); ignored by validate/resolve/codegen. |
+| `--pipe <pipe_code>` | The bundle has no `main_pipe`, or you want a different pipe. Used by execute/start (as `pipe_code`, exactly as given, so write `domain.pipe_code` when another bundle of the tree declares the same code); ignored by validate/resolve/codegen. |
| `--callback-url <url>` | (start only) Webhook the async result is POSTed to. Repeatable. Falls back to `CALLBACK_URL` in env/`.env`. |
| `--base-url` / `--token` | Target server + bearer for `--run`/`--curl`. |
| `--name <folder>` | Override the per-bundle Postman subfolder name. |
@@ -185,16 +189,16 @@ Useful options (apply across modes):
## Resolution rules (mirrors `pipelex run bundle`)
-- **Directory** → use `bundle.mthds` if present, else the single `*.mthds` in the dir. Multiple `.mthds` and no `bundle.mthds` is ambiguous → ask the user to pass the file directly. The sibling `inputs.json` is auto-detected. Every `*.mthds` in the directory is sent (main one first), so multi-file bundles resolve.
-- **`.mthds` file** → used as-is; inputs only come from `--inputs`.
-- **`pipe_code`** (run routes) comes from the bundle's `main_pipe` (override with `--pipe`).
-- **`inputs`** is the inputs.json content copied verbatim — exactly what the API receives, same as the CLI.
+- **Directory** → the main bundle is `bundle.mthds` if present, else the single `*.mthds` in the dir. Multiple `.mthds` and no `bundle.mthds` is ambiguous → ask the user to pass the file directly. The sibling `inputs.json` is auto-detected. The directory's whole method tree is sent as `files`, its bundles in subdirectories included, so multi-file methods resolve.
+- **`.mthds` file** → the main bundle is that file, and the tree it belongs to is sent: the package's tree when a `METHODS.toml` sits at or above its directory, else that one file. Inputs only come from `--inputs`.
+- **`pipe_code`** (run routes) is the main bundle's `main_pipe`, qualified by its domain (override with `--pipe`, sent as given).
+- **`inputs`** is the inputs.json content copied verbatim — exactly what the API receives, same as the CLI. The file itself is never one of the `files`.
## Important: file/document inputs are out of scope
We are not doing uploads. If inputs.json references a **local file** (e.g. `"url": "inputs/cv.pdf"`), the script copies it verbatim and prints a warning listing those URLs. Pass that warning on to the user: those entries must be replaced with real `https://` URLs before the request will run. Prefer demoing on a self-contained, text-input bundle.
-A related limit: the API receives only the inline bundle text, not the bundle directory. Bundles that depend on Python structure classes in a `structures/` folder (rather than inline `[concept.X.structure]` blocks) won't be fully self-contained over the API — flag that if you see a `structures/` dir doing real work.
+The method's own files are a different matter: a template, an image or a module the steps read travels in `files` with the bundles. Python is the one limit: a tree holding a `.py` file runs only on a sandbox-hosted runner (any other answers a `501`), and a Python structure class is refused on every runner, so a method whose concepts rely on a `structures/` folder of Python classes, rather than inline `[concept.X.structure]` blocks, will not run over the API — flag that if you see a `structures/` dir doing real work.
## If something's missing
@@ -203,6 +207,8 @@ A related limit: the API receives only the inline bundle text, not the bundle di
- **`codegen` without `--target`** → the script exits listing the three targets; ask the user which consumer they're generating for.
- **`start` selected via `--run`/`--curl` and no callback URL** → the script exits. Resolve from `--callback-url`, `CALLBACK_URL` in `.env`, or ask the user (e.g. a `https://webhook.site/...` endpoint). (The Postman push never hits this — it uses the `{{callback_url}}` variable and needs no resolution.)
- **No `main_pipe`, validating/resolving** → fine, not an error: those endpoints need no pipe; validate just returns `graph_spec: null`.
+- **The tree cannot be listed** (a symbolic link or a name the standard refuses inside it, a malformed `METHODS.toml`, a tree past the runner's limits) or **the main bundle is outside its own tree** (`.gitignore`d, say) → the script exits naming the cause; fix the method's directory, or point at the method's own folder rather than a directory above it.
+- **`ModuleNotFoundError: mthds` or `pipelex_api`** → the script ran with the wrong interpreter: use `.venv/bin/python` from `api/` (`make install` creates it) or a `make bundle-*` target.
## Constants
diff --git a/api/.claude/skills/postman-bundle/scripts/build_postman_query.py b/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
index 3560c71c0..7c067bf9d 100644
--- a/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
+++ b/api/.claude/skills/postman-bundle/scripts/build_postman_query.py
@@ -1,11 +1,31 @@
-#!/usr/bin/env python3
"""Build a runnable Postman request from a Pipelex MTHDS bundle and push it live.
Mirrors ``pipelex run bundle <path>`` resolution: given a bundle directory (or a
-single ``.mthds`` file), it finds the bundle file(s), extracts ``main_pipe``,
-loads the sibling ``inputs.json``, and inserts a ready-to-run request into the
+single ``.mthds`` file), it finds the main bundle file, extracts ``main_pipe``
+(sent as ``pipe_code``, qualified by that bundle's domain unless ``--pipe``
+names one), loads the sibling ``inputs.json``, and inserts a ready-to-run request into the
live "Pipelex FastAPI" Postman collection under ``Bundles/<bundle>/``.
+The method travels in every body as ``files``, the MTHDS standard's map of file
+entries: each file of the method tree the bundle belongs to, keyed by its tree
+path, its value the file's UTF-8 text, or ``{"base64": ...}`` for a file that is
+not UTF-8. The tree is listed by the ``mthds`` package's own lister, so it is
+judged by the membership rule the runner and every MTHDS client apply
+(``mthds/docs/spec/method-tree.md``): its root is the nearest directory at or
+above the bundle holding ``METHODS.toml``, else the directory handed over, else,
+for a ``.mthds`` file named alone, the one-file tree of that file; dotfiles,
+``.gitignore``d files, virtual environments and nested packages stay out. The
+script itself drops a run's inputs file, any file named ``inputs.json`` or
+``inputs.toml`` at any depth, as the standard's run-inputs default does, since
+the ``mthds`` lister pinned here still lists it: the sibling ``inputs.json``
+travels as ``inputs`` and never as a file of the tree. A body built here never uses the by-hash entry
+(``{"sha256", "size"}``), which only resolves once the bytes are uploaded to the
+runner's blob store, so a Postman query or a curl command stays self-contained.
+The listing stops at the runner's own limits on a tree a request sends.
+
+Run it with the server's environment (``make bundle-*`` from ``api/``, or
+``.venv/bin/python`` there): it imports ``mthds`` and ``pipelex_api.limits``.
+
Requests are generated per bundle (configurable via ``--endpoint``):
``Execute (sync)`` -> ``POST /v1/execute``, ``Start (async)`` ->
``POST /v1/start``, ``Validate (dry-run)`` -> ``POST /v1/validate``,
@@ -21,9 +41,9 @@ request fails fast (asking for one) if none is found.
File/document inputs are NOT uploaded. Any local (non-http) ``url`` in
inputs.json is copied verbatim and reported so you can swap in a real URL before
-running. Run this against a self-contained bundle (concepts/structures declared
-inline in the .mthds) — the API only receives the inline mthds_contents, not the
-directory, so sibling Python structure classes are not available.
+running. A tree holding a ``.py`` file runs only on a sandbox-hosted runner
+(elsewhere a 501), and a Python structure class is refused everywhere, so keep
+concepts and structures declared in the ``.mthds``.
"""
from __future__ import annotations
@@ -34,30 +54,42 @@ import os
import re
import sys
import tempfile
+import tomllib
import urllib.error
import urllib.request
from pathlib import Path
-from typing import Any, NoReturn, cast
+from typing import Any, NamedTuple, NoReturn, cast
+
+from mthds.method_tree.exceptions import MethodTreeError
+from mthds.method_tree.file_entry import encode_file_map
+from mthds.method_tree.lister import list_method_tree
+from mthds.method_tree.models import MethodTree, TreeLimits
+from mthds.method_tree.tree_path import nfc
+from pipelex.pipeline.default_file_names import DEFAULT_INPUTS_FILE_NAME, DEFAULT_INPUTS_TOML_FILE_NAME
+
+from pipelex_api.limits import MAX_METHOD_TREE_FILE_BYTES, MAX_METHOD_TREE_FILES, MAX_METHOD_TREE_TOTAL_BYTES
+
+# One file of a method tree as a body carries it: its UTF-8 text, or {"base64": ...} for other bytes.
+FileEntry = str | dict[str, str | int]
-try:
- import tomllib
-except ModuleNotFoundError: # Python < 3.11 — fall back to regex extraction.
- tomllib = None # type: ignore[assignment]
+# A run's inputs file, which the standard's run-inputs default keeps out of every tree at any depth
+# (`mthds/docs/spec/method-tree.md`, "Run Inputs Files"): a file named exactly so, letter case included.
+# A directory of either name excludes nothing.
+RUN_INPUTS_FILE_NAMES = frozenset({DEFAULT_INPUTS_FILE_NAME, DEFAULT_INPUTS_TOML_FILE_NAME})
DEFAULT_COLLECTION_UID = "35082494-559c5753-885c-409a-af63-7647fe28d301"
POSTMAN_API_BASE = "https://api.getpostman.com"
DEFAULT_BUNDLE_FILE_NAME = "bundle.mthds"
-DEFAULT_INPUTS_FILE_NAME = "inputs.json"
MTHDS_EXTENSION = ".mthds"
TOP_FOLDER_NAME = "Bundles"
# name -> (request display name, URL path segments, body kind)
-# Body kind selects the request shape:
-# "run" -> {pipe_code, mthds_contents, inputs} for the sync /execute
+# Body kind selects the request shape; every one carries the method as `files`:
+# "run" -> {pipe_code, inputs, files} for the sync /execute
# "start" -> the same plus {callback_urls} for the async /start
-# "validate" -> {mthds_contents, allow_signatures [, mthds_sources, render]}
-# "resolve" -> {files: [{content, source}]} — the crate envelope
-# "codegen" -> the crate envelope plus {kind, target}
+# "validate" -> {allow_signatures [, render], files}
+# "resolve" -> {files}
+# "codegen" -> {kind, target, files}
# Only "run"/"start" trigger inference; every other kind is a free
# parse/load/dry-run on the server.
ENDPOINTS: dict[str, tuple[str, list[str], str]] = {
@@ -70,7 +102,7 @@ ENDPOINTS: dict[str, tuple[str, list[str], str]] = {
# Which endpoints need what. execute/start run a specific pipe with real inputs;
# validate and the crate routes (resolve/codegen) derive everything from the
-# bundle text.
+# method's files.
PIPE_CODE_ENDPOINTS = {"execute", "start"}
INPUTS_ENDPOINTS = {"execute", "start"}
SIGNATURE_ENDPOINTS = {"validate"}
@@ -87,13 +119,23 @@ def fail(msg: str) -> NoReturn:
# --- bundle resolution (mirrors `pipelex run bundle`) ---------------------
-def resolve_bundle(raw_path: str, inputs_override: str | None) -> tuple[Path, list[Path], Path | None]:
- """Resolve a bundle path into (main_file, all_mthds_files, inputs_path).
+class ResolvedBundle(NamedTuple):
+ """What a bundle path resolves to: the path handed over, its main bundle file, and the inputs file to send."""
+
+ handed_over: Path
+ main_file: Path
+ inputs_path: Path | None
+
+
+def resolve_bundle(raw_path: str, inputs_override: str | None) -> ResolvedBundle:
+ """Resolve a bundle path into the path handed over, its main bundle file, and its inputs file.
Directory mode auto-detects ``bundle.mthds`` (or the single ``*.mthds``) and
a sibling ``inputs.json``, exactly like the CLI. File mode takes the .mthds
as-is and only uses ``--inputs`` if given (the CLI does not auto-detect
- inputs in file mode).
+ inputs in file mode). The main bundle file gives ``main_pipe`` and the
+ domain; what the request sends is the method tree of the path handed over
+ (``list_bundle_tree``).
"""
target = Path(raw_path).expanduser()
@@ -114,10 +156,6 @@ def resolve_bundle(raw_path: str, inputs_override: str | None) -> tuple[Path, li
)
main_file = mthds_files[0]
- # Send every .mthds in the directory (main first), mirroring how the CLI
- # adds the directory as a library dir so sibling bundles resolve.
- all_mthds = [main_file] + [path for path in sorted(target.glob(f"*{MTHDS_EXTENSION}")) if path != main_file]
-
inputs_path: Path | None = None
if inputs_override:
inputs_path = Path(inputs_override).expanduser()
@@ -125,32 +163,95 @@ def resolve_bundle(raw_path: str, inputs_override: str | None) -> tuple[Path, li
candidate = target / DEFAULT_INPUTS_FILE_NAME
if candidate.is_file():
inputs_path = candidate
- return main_file, all_mthds, inputs_path
+ return ResolvedBundle(handed_over=target, main_file=main_file, inputs_path=inputs_path)
if target.is_file() and target.suffix == MTHDS_EXTENSION:
inputs_path = Path(inputs_override).expanduser() if inputs_override else None
- return target, [target], inputs_path
+ return ResolvedBundle(handed_over=target, main_file=target, inputs_path=inputs_path)
fail(f"'{raw_path}' is not a {MTHDS_EXTENSION} file or directory")
+# --- the method tree, sent as `files` ----------------------------------------
+
+
+def list_bundle_tree(handed_over: Path) -> MethodTree:
+ """List the method tree of a bundle directory or a ``.mthds`` file, as the runner and every MTHDS client list it.
+
+ The listing is ``mthds``'s own, so its root and its membership rule are the
+ standard's, and it stops before reading any file once the tree passes one of
+ the runner's limits on a tree a request sends.
+ """
+ limits = TreeLimits(
+ max_files=MAX_METHOD_TREE_FILES,
+ max_total_bytes=MAX_METHOD_TREE_TOTAL_BYTES,
+ max_file_bytes=MAX_METHOD_TREE_FILE_BYTES,
+ )
+ try:
+ return list_method_tree(handed_over, limits=limits)
+ except MethodTreeError as exc:
+ fail(f"cannot list the method tree of '{handed_over}': {exc.message}")
+
+
+def tree_path_of(tree: MethodTree, file_path: Path) -> str | None:
+ """The tree path of a file on disk in a tree listed from disk, or None when the tree does not hold it."""
+ if tree.directory is None:
+ return None
+ try:
+ relative = file_path.resolve().relative_to(tree.directory)
+ except ValueError:
+ return None
+ tree_path = nfc(relative.as_posix())
+ return tree_path if tree_path in tree.members else None
+
+
+def is_run_inputs_file(tree_path: str) -> bool:
+ """Whether a file of a tree is a run's inputs file, which the standard's run-inputs default keeps out of every tree."""
+ return tree_path.rsplit("/", 1)[-1] in RUN_INPUTS_FILE_NAMES
+
+
+def build_files_map(tree: MethodTree) -> dict[str, FileEntry]:
+ """The ``files`` a body carries: every file of the tree under its tree path, inline, but a run's inputs file.
+
+ The ``mthds`` lister pinned here still lists a file named ``inputs.json`` or
+ ``inputs.toml``, which the standard keeps out of every tree and a runner
+ applying the rule refuses as a key, so it is dropped here: a run's inputs
+ travel as ``inputs``, never as a file of the method. Once the lister applies
+ the rule itself, no such file reaches this loop.
+
+ ``mthds`` encodes each file: its text when its bytes are UTF-8, else
+ ``{"base64": ...}``. No file is sent by hash, since a hash entry resolves
+ only in a runner's blob store, which a Postman query or a curl command never
+ uploads to.
+ """
+ contents: dict[str, bytes] = {}
+ for tree_path in tree.tree_paths:
+ if is_run_inputs_file(tree_path):
+ continue
+ location = tree.file_location(tree_path)
+ try:
+ contents[tree_path] = location.read_bytes()
+ except OSError as exc:
+ fail(f"cannot read '{location}' to send it as '{tree_path}': {exc}")
+ return encode_file_map(contents)
+
+
def parse_bundle_meta(text: str) -> tuple[str | None, str | None]:
"""Extract (main_pipe, domain) from a .mthds bundle.
- Uses tomllib when available (the .mthds top level is valid TOML), falling
- back to a line-anchored regex so a non-standard bundle still yields a guess.
+ Uses tomllib (the .mthds top level is valid TOML), falling back to a
+ line-anchored regex so a non-standard bundle still yields a guess.
"""
- if tomllib is not None:
- try:
- data = tomllib.loads(text)
- main_pipe = data.get("main_pipe")
- domain = data.get("domain")
- return (
- main_pipe if isinstance(main_pipe, str) else None,
- domain if isinstance(domain, str) else None,
- )
- except tomllib.TOMLDecodeError:
- pass
+ try:
+ data = tomllib.loads(text)
+ main_pipe = data.get("main_pipe")
+ domain = data.get("domain")
+ return (
+ main_pipe if isinstance(main_pipe, str) else None,
+ domain if isinstance(domain, str) else None,
+ )
+ except tomllib.TOMLDecodeError:
+ pass
main_match = re.search(r'(?m)^\s*main_pipe\s*=\s*"([^"]+)"', text)
domain_match = re.search(r'(?m)^\s*domain\s*=\s*"([^"]+)"', text)
return (
@@ -159,6 +260,18 @@ def parse_bundle_meta(text: str) -> tuple[str | None, str | None]:
)
+def qualified_main_pipe(main_pipe: str | None, domain: str | None) -> str | None:
+ """The main bundle's ``main_pipe`` as the pipe ref ``domain.pipe_code``, or bare when the bundle declares no domain.
+
+ The body carries the whole method tree, so a bare code would be ambiguous
+ as soon as another bundle of the tree declares a pipe of the same code; the
+ domain-qualified ref names the main bundle's pipe whatever its siblings hold.
+ """
+ if not main_pipe or not domain:
+ return main_pipe
+ return f"{domain}.{main_pipe}"
+
+
# --- inputs ----------------------------------------------------------------
@@ -253,86 +366,83 @@ def read_dotenv_value(key: str) -> str | None:
# --- Postman request construction -----------------------------------------
+# Every body puts `files` last: it holds the whole method, so the fields a reader
+# tweaks in Postman (pipe_code, inputs, callback_urls, render) stay on top.
+
+
def build_run_body(
pipe_code: str | None,
- mthds_contents: list[str],
+ files: dict[str, FileEntry],
inputs: Any,
callback_urls: list[str] | None = None,
) -> str:
- """Body for /execute and /start: pipe_code + mthds_contents + inputs.
+ """Body for /execute and /start: pipe_code + inputs + the method's files.
- The async /start adds ``callback_urls`` — the webhook(s) the runner
- POSTs the finished result to. It is omitted for the sync /execute.
+ ``pipe_code`` names the pipe to run; without it the runner starts at the
+ ``main_pipe`` of the tree's root ``METHODS.toml``, else at the one
+ ``main_pipe`` its bundles declare. The async /start adds ``callback_urls`` —
+ the webhook(s) the runner POSTs the finished result to. It is omitted for the
+ sync /execute.
"""
body: dict[str, Any] = {}
if pipe_code:
body["pipe_code"] = pipe_code
- body["mthds_contents"] = mthds_contents
if inputs is not None:
body["inputs"] = inputs
if callback_urls:
body["callback_urls"] = callback_urls
+ body["files"] = files
return json.dumps(body, indent=2, ensure_ascii=False)
def build_validate_body(
- mthds_contents: list[str],
+ files: dict[str, FileEntry],
allow_signatures: bool,
*,
- mthds_sources: list[str] | None = None,
render: list[str] | None = None,
) -> str:
- """Body for /validate — the skill-faithful wire shape.
+ """Body for /validate.
The endpoint takes no ``pipe_code`` and no ``inputs`` — it parses, loads, and
dry-runs every pipe with mock inputs and zero inference, so it needs only the
- bundle text. This mirrors exactly what ``mthds-agent validate bundle <file>``
- sends through ``--runner api`` (see ``mthds-js`` ``MthdsApiClient.validate``):
+ method's files. Each file is named by its tree path, which is what a
+ server-side ``errors[].location.file`` names.
- - ``mthds_contents`` — always.
- ``allow_signatures`` — always present (real clients send the bool verbatim;
``false`` is the strict default).
- - ``mthds_sources`` — when provided, parallel to ``mthds_contents``; each entry
- names the owning file so server-side ``errors[].location.file`` resolves.
- ``render`` — the opt-in Pipelex-API presentation hint (e.g. ``["markdown"]``).
A **skill** validates with the default Markdown view, so it sends
``render: ["markdown"]`` and reads the ``rendered_markdown`` the 200 carries; a
**hook** validates with ``--format json`` and omits ``render`` (lean structured
body). Pass ``--render markdown`` to reproduce the skill flow.
+ - ``files`` — always.
"""
- body: dict[str, Any] = {
- "mthds_contents": mthds_contents,
- "allow_signatures": allow_signatures,
- }
- if mthds_sources:
- body["mthds_sources"] = mthds_sources
+ body: dict[str, Any] = {"allow_signatures": allow_signatures}
if render:
body["render"] = render
+ body["files"] = files
return json.dumps(body, indent=2, ensure_ascii=False)
def build_crate_body(
- mthds_contents: list[str],
- mthds_sources: list[str],
+ files: dict[str, FileEntry],
*,
codegen_kind: str | None = None,
codegen_target: str | None = None,
) -> str:
- """Body for /resolve and /codegen — the crate routes' ``files[]`` envelope.
-
- Unlike validate's parallel ``mthds_contents[]``/``mthds_sources[]`` lists, the
- crate routes pair each content with its source in one ``{content, source}``
- entry (the shape the codegen spec pins). No ``pipe_code``, no ``inputs`` —
- the closure is the whole request. /codegen adds the two projection axes:
- ``kind`` (only ``types`` is served — an inputs template is projected client-side) and
- ``target`` (``ts-zod`` | ``python-pydantic`` | ``python-structures``).
+ """Body for /resolve and /codegen: the method's files, and nothing else for /resolve.
+
+ No ``pipe_code``, no ``inputs`` — the closure is the whole request. /codegen
+ adds the two projection axes: ``kind`` (only ``types`` is served — an inputs
+ template is projected client-side) and ``target`` (``ts-zod`` |
+ ``python-pydantic`` | ``python-structures``).
"""
body: dict[str, Any] = {}
if codegen_kind:
body["kind"] = codegen_kind
if codegen_target:
body["target"] = codegen_target
- body["files"] = [{"content": content, "source": source} for content, source in zip(mthds_contents, mthds_sources)]
+ body["files"] = files
return json.dumps(body, indent=2, ensure_ascii=False)
@@ -458,13 +568,17 @@ def pretty_json(text: str) -> str:
# --- main ------------------------------------------------------------------
-def main() -> None:
- parser = argparse.ArgumentParser(
- description="Resolve a Pipelex MTHDS bundle and either push a Postman query, emit curl, or run it."
- )
+def main(argv: list[str] | None = None) -> None:
+ parser = argparse.ArgumentParser(description="Resolve a Pipelex MTHDS bundle and either push a Postman query, emit curl, or run it.")
parser.add_argument("bundle", help="Path to a bundle directory or a .mthds file")
parser.add_argument("--inputs", help="Override inputs JSON path (default: auto-detect inputs.json in a directory)")
- parser.add_argument("--pipe", help="Override main_pipe (sets pipe_code). Ignored by --endpoint validate.")
+ parser.add_argument(
+ "--pipe",
+ help=(
+ "Override main_pipe (sets pipe_code, sent exactly as given: qualify it as domain.pipe_code when another "
+ "bundle of the tree declares the same code). Ignored by every endpoint but execute/start."
+ ),
+ )
parser.add_argument(
"--callback-url",
action="append",
@@ -523,7 +637,7 @@ def main() -> None:
# Only used by --run / --curl.
parser.add_argument("--base-url", default=DEFAULT_BASE_URL, help=f"API base URL for --run/--curl (default: {DEFAULT_BASE_URL})")
parser.add_argument("--token", help="Bearer token for --run/--curl (omit when the server runs AUTH_MODE=none)")
- args = parser.parse_args()
+ args = parser.parse_args(argv)
if sum([args.dry_run, args.curl, args.run]) > 1:
fail("choose at most one of --dry-run, --curl, --run.")
@@ -554,18 +668,34 @@ def main() -> None:
if "codegen" in active_keys and not args.target:
fail(f"--endpoint codegen requires --target <{'|'.join(CODEGEN_TARGETS)}>.")
- main_file, all_mthds, inputs_path = resolve_bundle(args.bundle, args.inputs)
+ bundle = resolve_bundle(args.bundle, args.inputs)
+ main_file = bundle.main_file
+ inputs_path = bundle.inputs_path
main_text = main_file.read_text(encoding="utf-8")
main_pipe, domain = parse_bundle_meta(main_text)
- # `pipe_code` is required only by the run endpoints — /validate derives
- # everything it needs from the bundle text and takes no pipe_code.
- pipe_code = args.pipe or main_pipe
- if needs_pipe and not pipe_code:
+ # The method travels as its whole tree. The main bundle must be one of its
+ # files, or the pipe_code taken from it would name a pipe the runner never
+ # receives.
+ tree = list_bundle_tree(bundle.handed_over)
+ main_tree_path = tree_path_of(tree, main_file)
+ if main_tree_path is None:
fail(
- f"could not determine main_pipe from '{main_file}'. Declare main_pipe in the bundle, "
- "or pass --pipe <pipe_code>."
+ f"the membership rule keeps '{main_file}' out of the method tree rooted at '{tree.directory}', so the "
+ "runner would never receive it. Check the tree's .gitignore files and the [package] exclude patterns of "
+ "its METHODS.toml."
)
+ files = build_files_map(tree)
+ tree_root_label = f"{tree.directory} (the one-file tree of {main_tree_path})" if tree.is_one_file else f"{tree.directory}"
+
+ # `pipe_code` is required only by the run endpoints — /validate derives
+ # everything it needs from the method's files and takes no pipe_code. An
+ # explicit --pipe is sent exactly as given; the inferred one is qualified with
+ # the main bundle's domain, since the whole tree travels and another of its
+ # bundles may declare a pipe of the same bare code.
+ pipe_code = args.pipe or qualified_main_pipe(main_pipe, domain)
+ if needs_pipe and not pipe_code:
+ fail(f"could not determine main_pipe from '{main_file}'. Declare main_pipe in the bundle, or pass --pipe <pipe_code>.")
# callback_urls belongs to the async /start endpoint only. In the Postman
# push it is always the {{callback_url}} collection variable — switchable
@@ -586,8 +716,6 @@ def main() -> None:
"have one, ask the user for a callback URL (e.g. a https://webhook.site/... endpoint)."
)
- mthds_contents = [path.read_text(encoding="utf-8") for path in all_mthds]
-
# inputs (and their local-url warnings) matter only to the run endpoints;
# every other endpoint ignores them, so don't load or warn otherwise.
inputs: Any = None
@@ -597,13 +725,9 @@ def main() -> None:
subfolder_name = args.name or domain or main_file.stem
- # Per-file sources, parallel to mthds_contents. /validate sends them as the
- # `mthds_sources` list; the crate routes pair them into `files[].source`.
- # Either way, server-side diagnostics name the owning file. `render` is the
- # opt-in presentation hint on /validate: a skill validates with the default
- # Markdown view (`--render markdown`), a hook omits it and reads the
- # structured body.
- mthds_sources = [path.name for path in all_mthds]
+ # `render` is the opt-in presentation hint on /validate: a skill validates
+ # with the default Markdown view (`--render markdown`), a hook omits it and
+ # reads the structured body.
render_formats: list[str] = []
for raw_format in cast("list[str]", args.render or []):
normalized = raw_format.strip().lower()
@@ -614,23 +738,20 @@ def main() -> None:
# execute/start which share the run-request shape via "run"/"start").
bodies: dict[str, str] = {}
if needs_run:
- bodies["run"] = build_run_body(pipe_code, mthds_contents, inputs)
+ bodies["run"] = build_run_body(pipe_code, files, inputs)
if needs_start:
- bodies["start"] = build_run_body(pipe_code, mthds_contents, inputs, callback_urls=callback_urls)
+ bodies["start"] = build_run_body(pipe_code, files, inputs, callback_urls=callback_urls)
if needs_validate:
- bodies["validate"] = build_validate_body(
- mthds_contents,
- args.allow_signatures,
- mthds_sources=mthds_sources,
- render=render_formats or None,
- )
+ bodies["validate"] = build_validate_body(files, args.allow_signatures, render=render_formats or None)
if "resolve" in active_keys:
- bodies["resolve"] = build_crate_body(mthds_contents, mthds_sources)
+ bodies["resolve"] = build_crate_body(files)
if "codegen" in active_keys:
- bodies["codegen"] = build_crate_body(mthds_contents, mthds_sources, codegen_kind="types", codegen_target=args.target)
+ bodies["codegen"] = build_crate_body(files, codegen_kind="types", codegen_target=args.target)
+ file_labels = [f"{tree_path} (base64)" if isinstance(entry, dict) else tree_path for tree_path, entry in files.items()]
print(f"Bundle: {main_file}")
- print(f"mthds files: {', '.join(path.name for path in all_mthds)}")
+ print(f"tree root: {tree_root_label}")
+ print(f"files: {', '.join(file_labels)}")
if needs_pipe:
print(f"pipe_code: {pipe_code}")
if needs_inputs:
@@ -641,10 +762,10 @@ def main() -> None:
print(f"signatures: allow_signatures={args.allow_signatures}")
if needs_validate:
render_note = ", ".join(render_formats) or "none (structured-only, hook-style)"
- print(f"validate: POST /v1/validate (no inference) — sources: {', '.join(mthds_sources)}")
+ print("validate: POST /v1/validate (no inference)")
print(f" render: {render_note}")
if needs_crate:
- print(f"crate: files[] envelope — sources: {', '.join(mthds_sources)} (no inference)")
+ print("crate: the method's files, nothing else (no inference)")
if "codegen" in active_keys:
print(f"codegen: kind=types target={args.target}")
if local_url_warnings:
@@ -683,11 +804,8 @@ def main() -> None:
# mode == "postman"
assert api_key is not None # guaranteed above for the postman path
- trailer = (
- "Generated by the postman-bundle skill. Set `base_url` and `auth_token` in your "
- "Postman environment before running."
- )
- source_lines = f"- Source: `{main_file}`\n- .mthds files sent: {len(mthds_contents)}\n"
+ trailer = "Generated by the postman-bundle skill. Set `base_url` and `auth_token` in your Postman environment before running."
+ source_lines = f"- Source: `{main_file}`\n- Method tree: `{tree_root_label}`, sent as `files`: {', '.join(file_labels)}\n"
def describe(key: str) -> str:
"""Per-endpoint Postman request description (shown in the collection)."""
@@ -700,7 +818,7 @@ def main() -> None:
return (
f"Run the `{subfolder_name}` bundle (mirrors `pipelex run bundle`). Triggers REAL inference.\n\n"
f"{source_lines}"
- f"- main_pipe: `{pipe_code}`\n"
+ f"- pipe_code: `{pipe_code}`\n"
f"- inputs: {inputs_path or 'none'}\n"
f"{callback_note}\n{trailer}"
)
@@ -714,7 +832,6 @@ def main() -> None:
f"Validate (dry-run) the `{subfolder_name}` bundle — parse, load, and dry-run every pipe "
"with NO inference (free, no LLM cost).\n\n"
f"{source_lines}"
- f"- mthds_sources: `{mthds_sources}`\n"
f"- allow_signatures: {args.allow_signatures}\n"
f"{render_line}\n"
"Takes no pipe_code and no inputs. 200 discriminated on `is_valid`: the bundle blueprint, "
diff --git a/api/Makefiles/Makefile.local.mk b/api/Makefiles/Makefile.local.mk
index 26870fcd3..837ede74049 100644
--- a/api/Makefiles/Makefile.local.mk
+++ b/api/Makefiles/Makefile.local.mk
@@ -195,8 +195,10 @@ tstop: temporal-stop
# Resolve an MTHDS bundle the same way `pipelex run bundle <path>` does and send
# it to the API as a request — run it directly, dry-run validate it, resolve it
# to its normalized crate, project typed codegen artifacts, emit curl, push a
-# Postman query, or just print the body. Runs the skill's helper script with OUR
-# venv python.
+# Postman query, or just print the body. The method travels as `files`, every
+# file of its tree. Runs the skill's helper script with OUR venv python, which
+# the targets install first: the script lists the tree with the `mthds` package
+# and reads the runner's tree limits from `pipelex_api`.
#
# make bundle-run BUNDLE=../../pipelex-demos/mthds-wip/fashion_moodboard
# make bundle-validate BUNDLE=../../pipelex-demos/mthds-wip/fashion_moodboard
@@ -225,7 +227,7 @@ BUNDLE_RUN_OPTS = $(if $(BASE_URL),--base-url $(BASE_URL)) $(if $(TOKEN),--token
check-bundle-arg:
@test -n "$(BUNDLE)" || { echo "ERROR: set BUNDLE=<bundle dir or .mthds file>, e.g. make bundle-run BUNDLE=../../pipelex-demos/mthds-wip/fashion_moodboard"; exit 1; }
-bundle-run: env check-bundle-arg
+bundle-run: install check-bundle-arg
$(call PRINT_TITLE,"Running bundle against the API")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --run $(BUNDLE_RUN_OPTS) $(BUNDLE_OPTS)
@@ -233,32 +235,32 @@ bundle-run: env check-bundle-arg
# pass ENDPOINT here). NAME/ALLOW_SIGNATURES/RENDER/BASE_URL/TOKEN/ARGS still
# apply (RENDER=markdown reproduces a skill-driven validate); PIPE/INPUTS are
# intentionally omitted — the validate endpoint ignores them.
-bundle-validate: env check-bundle-arg
+bundle-validate: install check-bundle-arg
$(call PRINT_TITLE,"Validating bundle via the API - dry-run with no inference")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --run --endpoint validate $(if $(ALLOW_SIGNATURES),--allow-signatures) $(if $(RENDER),--render $(RENDER)) $(if $(NAME),--name $(NAME)) $(BUNDLE_RUN_OPTS) $(ARGS)
# Resolve the bundle's closure to its normalized library crate via /v1/resolve —
# no inference, no dry-run sweep, no cost. PIPE/INPUTS are intentionally
# omitted — the crate routes take neither.
-bundle-resolve: env check-bundle-arg
+bundle-resolve: install check-bundle-arg
$(call PRINT_TITLE,"Resolving bundle to its normalized crate via the API")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --run --endpoint resolve $(if $(NAME),--name $(NAME)) $(BUNDLE_RUN_OPTS) $(ARGS)
# Project typed artifacts + codegen.lock via /v1/codegen — no inference, no
# cost. TARGET is required: ts-zod | python-pydantic | python-structures.
-bundle-codegen: env check-bundle-arg
+bundle-codegen: install check-bundle-arg
@test -n "$(TARGET)" || { echo "ERROR: set TARGET=<ts-zod|python-pydantic|python-structures>, e.g. make bundle-codegen BUNDLE=... TARGET=ts-zod"; exit 1; }
$(call PRINT_TITLE,"Generating typed artifacts via the API codegen route")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --run --endpoint codegen --target $(TARGET) $(if $(NAME),--name $(NAME)) $(BUNDLE_RUN_OPTS) $(ARGS)
-bundle-curl: env check-bundle-arg
+bundle-curl: install check-bundle-arg
$(call PRINT_TITLE,"Emitting curl for bundle")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --curl $(BUNDLE_RUN_OPTS) $(BUNDLE_OPTS)
-bundle-postman: env check-bundle-arg
+bundle-postman: install check-bundle-arg
$(call PRINT_TITLE,"Pushing Postman query for bundle")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) $(BUNDLE_OPTS)
-bundle-dry: env check-bundle-arg
+bundle-dry: install check-bundle-arg
$(call PRINT_TITLE,"Bundle request body - dry run only")
$(VENV_PYTHON) $(BUNDLE_SCRIPT) $(BUNDLE) --dry-run $(BUNDLE_OPTS)
diff --git a/api/tests/unit/test_postman_bundle_script.py b/api/tests/unit/test_postman_bundle_script.py
new file mode 100644
index 000000000..f47acf05e
--- /dev/null
+++ b/api/tests/unit/test_postman_bundle_script.py
@@ -0,0 +1,252 @@
+"""The `postman-bundle` skill sends a method as `files`: every file of its tree, inline, and none of the retired fields.
+
+`build_postman_query.py` is the skill's standalone script, not a package module, so it is loaded from its path. Each
+body is built as the script's preview (`--dry-run`) prints it, never pushed to Postman nor sent over the network; the
+runner's own routes then read the printed body in process, so a body the skill builds is one the runner accepts.
+"""
+
+import base64
+import importlib.util
+import json
+from pathlib import Path
+from types import ModuleType
+from typing import Any
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+from pytest_mock import MockerFixture
+
+from pipelex_api.api_config import ApiConfig
+from pipelex_api.exception_handlers import register_exception_handlers
+from pipelex_api.routes import router as api_router
+
+_MEMBER_ROOT = Path(__file__).resolve().parents[2]
+_SCRIPT_PATH = _MEMBER_ROOT / ".claude" / "skills" / "postman-bundle" / "scripts" / "build_postman_query.py"
+
+
+def _load_script() -> ModuleType:
+ spec = importlib.util.spec_from_file_location("build_postman_query", _SCRIPT_PATH)
+ assert spec is not None
+ assert spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+ return module
+
+
+postman_script = _load_script()
+
+_GREET_MTHDS = """\
+domain = "greeting"
+main_pipe = "greet"
+
+[pipe.greet]
+type = "PipeCompose"
+description = "Greet"
+inputs = { text = "Text" }
+output = "Text"
+template = "Hello $text"
+"""
+
+# A sibling bundle of another domain declaring a pipe of the same bare code as the main bundle's `main_pipe`.
+_OTHER_MTHDS = """\
+domain = "other"
+
+[pipe.greet]
+type = "PipeCompose"
+description = "Greet otherwise"
+inputs = { text = "Text" }
+output = "Text"
+template = "Farewell $text"
+"""
+
+_CARD_HTML = "<p>Hello, {{ text }}</p>\n"
+_ICON = b"\x89PNG\r\n\x1a\n not text"
+_MANIFEST = """\
+[package]
+name = "greeting"
+address = "github.com/acme/greeting"
+version = "0.1.0"
+description = "A package whose root is above the bundle's directory."
+"""
+_README = "# Greeting\n"
+_INPUTS = {"text": "world"}
+_INPUTS_TOML = 'text = "world"\n'
+_EXAMPLE_INPUTS = '{"text": "an example for people to read"}\n'
+_CALLBACK_URL = "https://example.com/hook"
+
+# The tree of `bundle_dir`: its text inline, its binary file in Base64, and nothing the membership rule keeps out, a
+# run's inputs file at any depth included, while example inputs under another name are an ordinary member.
+_EXPECTED_FILES: dict[str, Any] = {
+ "assets/icon.png": {"base64": base64.b64encode(_ICON).decode("ascii")},
+ "bundle.mthds": _GREET_MTHDS,
+ "examples/greeting-inputs.json": _EXAMPLE_INPUTS,
+ "other.mthds": _OTHER_MTHDS,
+ "templates/card.html": _CARD_HTML,
+}
+
+_REQUEST_BODY_MARKER = "--- request body ("
+
+
[email protected](name="bundle_dir")
+def bundle_dir_fixture(tmp_path: Path) -> Path:
+ """A method's directory: two bundles, a template, a binary file and the run's inputs, beside files the membership rule keeps out."""
+ bundle_dir = tmp_path / "greeting"
+ for subdirectory in ("templates", "assets", "examples", "drafts", "__pycache__"):
+ (bundle_dir / subdirectory).mkdir(parents=True)
+ (bundle_dir / "bundle.mthds").write_text(_GREET_MTHDS, encoding="utf-8")
+ (bundle_dir / "other.mthds").write_text(_OTHER_MTHDS, encoding="utf-8")
+ (bundle_dir / "inputs.json").write_text(json.dumps(_INPUTS), encoding="utf-8")
+ (bundle_dir / "examples" / "inputs.toml").write_text(_INPUTS_TOML, encoding="utf-8")
+ (bundle_dir / "examples" / "greeting-inputs.json").write_text(_EXAMPLE_INPUTS, encoding="utf-8")
+ (bundle_dir / "templates" / "card.html").write_text(_CARD_HTML, encoding="utf-8")
+ (bundle_dir / "assets" / "icon.png").write_bytes(_ICON)
+ (bundle_dir / ".gitignore").write_text("drafts/\n", encoding="utf-8")
+ (bundle_dir / "drafts" / "old.mthds").write_text(_GREET_MTHDS, encoding="utf-8")
+ (bundle_dir / ".secret").write_text("kept out by its dot\n", encoding="utf-8")
+ (bundle_dir / "__pycache__" / "cache.pyc").write_bytes(b"\x00")
+ return bundle_dir
+
+
+def _previewed_body_text(capsys: pytest.CaptureFixture[str], *args: str) -> str:
+ """The JSON text of the one request body the script's preview prints for these arguments."""
+ postman_script.main([*args, "--dry-run"])
+ printed = capsys.readouterr().out
+ assert printed.count(_REQUEST_BODY_MARKER) == 1, printed
+ _, _, after_marker = printed.partition(_REQUEST_BODY_MARKER)
+ _, _, body_text = after_marker.partition("\n")
+ return body_text
+
+
+def _build_client(mocker: MockerFixture) -> TestClient:
+ mocker.patch(
+ "pipelex_api.routes.pipelex.pipeline.get_api_config",
+ return_value=ApiConfig(orchestration_mode="direct", allow_request_orchestration_mode_override=False),
+ )
+ app = FastAPI()
+ app.include_router(api_router, prefix="/v1")
+ register_exception_handlers(app)
+ return TestClient(app)
+
+
+class TestPostmanBundleScript:
+ def test_the_files_are_every_file_of_the_method_tree_inline(self, bundle_dir: Path) -> None:
+ """Text travels as text, other bytes in Base64; a dotfile, a `.gitignore`d file, a cache or a run's inputs file never travels."""
+ files = postman_script.build_files_map(postman_script.list_bundle_tree(bundle_dir))
+
+ assert files == _EXPECTED_FILES
+
+ def test_a_bundle_named_alone_sends_its_one_file_tree(self, bundle_dir: Path) -> None:
+ files = postman_script.build_files_map(postman_script.list_bundle_tree(bundle_dir / "bundle.mthds"))
+
+ assert files == {"bundle.mthds": _GREET_MTHDS}
+
+ def test_a_manifest_above_the_bundle_roots_the_tree(self, tmp_path: Path) -> None:
+ """The files are keyed by their paths from the tree's root, the directory of the nearest `METHODS.toml`."""
+ package_root = tmp_path / "package"
+ method_dir = package_root / "methods" / "greeting"
+ method_dir.mkdir(parents=True)
+ (package_root / "METHODS.toml").write_text(_MANIFEST, encoding="utf-8")
+ (package_root / "README.md").write_text(_README, encoding="utf-8")
+ (method_dir / "bundle.mthds").write_text(_GREET_MTHDS, encoding="utf-8")
+
+ files = postman_script.build_files_map(postman_script.list_bundle_tree(method_dir))
+
+ assert files == {"METHODS.toml": _MANIFEST, "README.md": _README, "methods/greeting/bundle.mthds": _GREET_MTHDS}
+
+ @pytest.mark.parametrize(
+ ("endpoint_args", "expected_fields"),
+ [
+ (["--endpoint", "execute"], {"pipe_code": "greeting.greet", "inputs": _INPUTS}),
+ (
+ ["--endpoint", "start", "--callback-url", _CALLBACK_URL],
+ {"pipe_code": "greeting.greet", "inputs": _INPUTS, "callback_urls": [_CALLBACK_URL]},
+ ),
+ (["--endpoint", "validate", "--render", "markdown"], {"allow_signatures": False, "render": ["markdown"]}),
+ (["--endpoint", "resolve"], {}),
+ (["--endpoint", "codegen", "--target", "ts-zod"], {"kind": "types", "target": "ts-zod"}),
+ ],
+ )
+ def test_every_body_carries_the_files_and_no_retired_field(
+ self,
+ capsys: pytest.CaptureFixture[str],
+ bundle_dir: Path,
+ endpoint_args: list[str],
+ expected_fields: dict[str, Any],
+ ) -> None:
+ """No `mthds_contents`, `mthds_sources`, `bundle_b64` or `resources`, and the whole method last, after what a reader edits.
+
+ The sibling `inputs.json` the script reads as the run's inputs, and the `inputs.toml` deeper in the tree, are in no
+ endpoint's `files`, whether or not the `mthds` lister keeps them out itself.
+ """
+ body = json.loads(_previewed_body_text(capsys, str(bundle_dir), *endpoint_args))
+
+ assert body == {**expected_fields, "files": _EXPECTED_FILES}
+ assert "inputs.json" not in body["files"]
+ assert "examples/inputs.toml" not in body["files"]
+ assert list(body)[-1] == "files"
+
+ @pytest.mark.parametrize(
+ ("pipe_args", "expected_pipe_code"),
+ [
+ ([], "greeting.greet"),
+ (["--pipe", "greet"], "greet"),
+ (["--pipe", "other.greet"], "other.greet"),
+ ],
+ )
+ def test_the_inferred_pipe_code_is_qualified_and_an_explicit_one_is_sent_as_given(
+ self,
+ capsys: pytest.CaptureFixture[str],
+ bundle_dir: Path,
+ pipe_args: list[str],
+ expected_pipe_code: str,
+ ) -> None:
+ """The tree carries `other.greet` beside `greeting.greet`, so the main bundle's `main_pipe` travels qualified by its domain."""
+ body = json.loads(_previewed_body_text(capsys, str(bundle_dir), "--endpoint", "execute", *pipe_args))
+
+ assert body["pipe_code"] == expected_pipe_code
+
+ def test_the_runner_runs_the_execute_body(self, capsys: pytest.CaptureFixture[str], mocker: MockerFixture, bundle_dir: Path) -> None:
+ """The sibling `inputs.json` is found, and the runner runs the main bundle's pipe, not its sibling's namesake."""
+ body_text = _previewed_body_text(capsys, str(bundle_dir), "--endpoint", "execute")
+ assert json.loads(body_text)["inputs"] == _INPUTS
+
+ response = _build_client(mocker).post("/v1/execute", content=body_text, headers={"Content-Type": "application/json"})
+
+ assert response.status_code == 200, response.text
+ run_result = response.json()
+ assert run_result["state"] == "COMPLETED"
+ assert "Hello world" in json.dumps(run_result["pipe_output"]["working_memory"])
+
+ @pytest.mark.parametrize(
+ ("endpoint_args", "route"),
+ [
+ (["--endpoint", "validate"], "/v1/validate"),
+ (["--endpoint", "resolve"], "/v1/resolve"),
+ (["--endpoint", "codegen", "--target", "python-pydantic"], "/v1/codegen"),
+ ],
+ )
+ def test_the_runner_reads_the_free_bodies_as_a_valid_method(
+ self,
+ capsys: pytest.CaptureFixture[str],
+ mocker: MockerFixture,
+ bundle_dir: Path,
+ endpoint_args: list[str],
+ route: str,
+ ) -> None:
+ body_text = _previewed_body_text(capsys, str(bundle_dir), *endpoint_args)
+
+ response = _build_client(mocker).post(route, content=body_text, headers={"Content-Type": "application/json"})
+
+ assert response.status_code == 200, response.text
+ assert response.json()["is_valid"] is True, response.text
+
+ def test_a_main_bundle_the_membership_rule_keeps_out_is_refused(self, capsys: pytest.CaptureFixture[str], bundle_dir: Path) -> None:
+ """A pipe_code taken from a bundle the runner would never receive is never sent."""
+ (bundle_dir / ".gitignore").write_text("drafts/\nbundle.mthds\n", encoding="utf-8")
+
+ with pytest.raises(SystemExit) as exit_info:
+ postman_script.main([str(bundle_dir), "--endpoint", "execute", "--dry-run"])
+
+ assert exit_info.value.code == 1
+ assert "the membership rule keeps" in capsys.readouterr().err