Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Lockstep-release-chain · L-260930-89f645 (#1309)
Louis Choquel1bf3c98cc73e
diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.mdindex 7000bc109..4adf8b676 100644--- a/.claude/skills/release/SKILL.md+++ b/.claude/skills/release/SKILL.md@@ -1,11 +1,13 @@---name: releasedescription: >- Cut a release of the pipelex Python package: the gates, the migration-ledger- cross-check, the CHANGELOG.md entry, the pyproject.toml bump with the lock and- the badge, one commit, and a pull request to main that publishes to PyPI on- merge. Use when the user says "release", "cut a release", "bump version",- "prepare a release", "make a release", "ship it", "create release branch",+ Cut a release of pipelex, which ships the pipelex and pipelex-api packages and+ the pipelex/pipelex-api Docker image under one version: the gates, the+ migration-ledger cross-check, the CHANGELOG.md entry, the pyproject.toml bump+ with the lock and the badge, one commit, and a pull request to main that+ publishes to PyPI and Docker Hub on merge. Use when the user says "release",+ "cut a release", "bump version", "prepare a release", "make a release",+ "ship it", "create release branch","promote dev to main", or any variation of shipping a new version of pipelex.Changelog content passed inline ("/release Added new extract backend") becomesthe entry. The merge is landed by /ledger-land, never by this skill.@@ -17,21 +19,32 @@ The procedure is the workspace release play, [`docs/workspace/releasing.md`](../## What ships+One version, three artifacts, all from one run of `.github/workflows/publish-pypi.yml`, in this order: the `pipelex` wheel, the GitHub Release that tags the release commit, the `pipelex-api` wheel, then the image. The server in `api/` takes the library's version, so nothing of it is bumped separately.+- **PyPI: `pipelex`.** `.github/workflows/publish-pypi.yml` builds and publishes through trusted publishing. It fires on the release pull request **closing merged into `main`** (`pull_request: types: [closed]`), not on the push to `main` — so the run is keyed to the release branch and its `headSha` is that branch's last commit, never the merge commit. Where the play says "the run on the merge SHA", read it here as the run on `release/vX.Y.Z`.+- **PyPI: `pipelex-api`.** The same workflow's `build-api` job builds the server's wheel and sdist and refuses them unless `api/scripts/check_lockstep_pin.py` finds `pipelex[...]==X.Y.Z` in both, the pin `api/hatch_build.py` writes, and `publish-to-pypi` waits for it, so a server that does not build stops the release before anything is uploaded; `publish-api-to-pypi` uploads them through trusted publishing in the same `pypi` environment, only after `github-release`, so the `pipelex` they pin is on PyPI and the tag guard has accepted the commit they were built from before a server file reaches PyPI.+- **Docker Hub: `pipelex/pipelex-api:X.Y.Z`, and `latest`.** The `publish-docker-hub` job calls `.github/workflows/publish-docker-hub.yml` with the version once both distributions are on PyPI: it checks out the commit the `vX.Y.Z` tag names, runs `make -C api deploy-docker-hub` there (build for `linux/amd64` from the repository root, smoke-test that the image boots and reports X.Y.Z for the server and its pipelex, push the version tag, and `latest` when X.Y.Z is the newest stable release), confirms the tags through Docker Hub's tags API, and pushes `api/README.md` as the Docker Hub overview. A pre-release pushes its own tag and leaves `latest` alone, and so does a retry of an older release.- **The GitHub Release and the `vX.Y.Z` tag.** The same workflow's `github-release` job creates them, and the tag exists only as a side effect of `gh release create`. The release is created and the dists attached *before* Sigstore signing, which is allowed to fail: an unsigned release is reported as a warning, not a failure, because the tag is the record of what shipped.-- **The documentation site.** `.github/workflows/deploy-docs.yml` fires on the push to `main` and runs `make docs-deploy-stable`, which deploys the version read from `pyproject.toml` with the `latest` alias and republishes the root sitemap.+- **The documentation site.** `.github/workflows/deploy-docs.yml` fires on the push to `main` and runs `make docs-deploy-stable`, which deploys the version read from `pyproject.toml` with the `latest` alias and republishes the root sitemap. The API server's pages are part of it, under `api-server/`, with the OpenAPI artifact beside them.-The landing verifies the publish from three places:+The landing verifies the publish from the run, the registries and the tag:```bashgh run list --workflow=publish-pypi.yml --branch release/vX.Y.Z --limit 3 --json name,conclusion,headSha,event,url # successpip index versions pipelex # the registry answers X.Y.Z+pip index versions pipelex-api # the registry answers X.Y.Z+curl -fsS https://hub.docker.com/v2/namespaces/pipelex/repositories/pipelex-api/tags/X.Y.Z | jq -r .digest # Docker Hub serves the tag+curl -fsS https://hub.docker.com/v2/namespaces/pipelex/repositories/pipelex-api/tags/latest | jq -r .digest # the same digest, for a stable versiongit -C <main> fetch --tags --prune origin && git -C <main> tag --list vX.Y.Z```+The run concludes `success` only when every job did, the image's included, so a green run already says all three artifacts shipped; the registry answers are what confirms it from the outside. `docker manifest inspect pipelex/pipelex-api:X.Y.Z` is the other way to read the image tag.+`gh release view vX.Y.Z` confirms the Release and its notes. A publish that failed *after* the merge is recovered by **re-running that run's failed jobs**, which keeps the successful ones, and that is the path to reach for. It is safe once the dists have reached PyPI: `skip-existing` opens on a re-run and on a dispatch, so the publish job no longer dies on the files already there, and the build job recovers the bytes this version published rather than rebuilding them — the artifact an attempt of the same run stored, else one an earlier run stored for the same commit, else the files PyPI serves — then refuses to go on unless what it holds is byte-for-byte what PyPI publishes. The Release step itself is idempotent: it edits an existing release rather than failing on one.-`workflow_dispatch` is the other path, needed when the failure calls for a change to the workflow file, since a re-run replays the file the run started with. **Once the release exists, a dispatch succeeds only from a ref whose head is the commit the release is tagged at** — the release pull request's base branch, `main` normally, while nothing further has landed on it. The tag names the merge commit, a dispatch ships the head of the ref it was dispatched from, and `github-release`'s tag guard refuses any other commit: a dispatch from `release/vX.Y.Z` is therefore always refused, although it is the ref the failed run is listed under. That dispatch recovers from PyPI rather than from an artifact, the release branch's build being keyed to a different commit, and the bytes are the same either way. A dispatch accepts `main`, `release/vX.Y.Z` and `pre-release/v*` and nothing else, and its run lists under the ref it was dispatched from rather than under the release branch.+The same goes for the server's two artifacts. A failed `publish-api-to-pypi` is re-run on its own, from the build its run stored. A failed `publish-docker-hub` is re-run on its own too: a version Docker Hub already serves is never built again, since a rebuild would be a different image under the same tag, so a retry that finds it there only moves `latest`, when the version is the newest stable release, and confirms the tags. When that failure needs a change to `publish-docker-hub.yml`, which a re-run would not pick up, dispatch it alone from the branch carrying the fix, `gh workflow run publish-docker-hub.yml --ref <branch> -f version=X.Y.Z`: whatever the ref, it builds the commit the `vX.Y.Z` tag names. The Dockerfile and the `deploy-docker-hub` recipe are that commit's too, so a fix there needs a new release. Docker Hub's tags API then confirms the push, as above. The overview step is allowed to fail: a warning in the run says so, usually because the token lacks the read/write/delete scope, and the cure is that scope and a re-run of the job, or pasting `api/README.md` into the overview on Docker Hub.++`workflow_dispatch` of `publish-pypi.yml` is the other path, needed when the failure calls for a change to the workflow file, since a re-run replays the file the run started with. **Once the release exists, a dispatch succeeds only from a ref whose head is the commit the release is tagged at** — the release pull request's base branch, `main` normally, while nothing further has landed on it. The tag names the merge commit, a dispatch ships the head of the ref it was dispatched from, and `github-release`'s tag guard refuses any other commit: a dispatch from `release/vX.Y.Z` is therefore always refused, although it is the ref the failed run is listed under. That dispatch recovers from PyPI rather than from an artifact, the release branch's build being keyed to a different commit, and the bytes are the same either way. A dispatch accepts `main`, `release/vX.Y.Z` and `pre-release/v*` and nothing else, and its run lists under the ref it was dispatched from rather than under the release branch.## Version files and the lock@@ -39,7 +52,7 @@ git -C <main> fetch --tags --prune origin && git -C <main> tag --list vX.Y.Z- **The lock** — `make li` (lock + install) regenerates `uv.lock`. Stop and report if it fails; `uv-lock-check` in CI fails the pull request over a stale lock.- **Also stamped:**- **`.badges/tests.json`** — set `"message"` to what `make test-count` prints, leaving every other field alone, then run `make check-test-badge` to confirm the two agree. A mismatch is a CI failure on the pull request.- - **`api/docs/openapi/pipelex-api.openapi.yaml`** — `make -C api openapi-export`. The API server in `api/` takes the library's version, and its committed OpenAPI artifact carries it as `info.version`, so the bump moves the artifact and `make agent-check`, whose `api-agent-check` runs `openapi-check`, fails until it is re-exported. The only change it should show is that one line; anything more is a wire change that belongs in the changelog.+ - **`docs/api-server/openapi/pipelex-api.openapi.yaml`** — `make -C api openapi-export`. The API server in `api/` takes the library's version, and its committed OpenAPI artifact carries it as `info.version`, so the bump moves the artifact and `make agent-check`, whose `api-agent-check` runs `openapi-check`, fails until it is re-exported. The only change it should show is that one line; anything more is a wire change that belongs in the changelog.- **`.test_durations`** — `make store-test-durations`, the per-test timing map `pytest-split` uses to balance the CI test shards. The refresh is incremental: it collects the suite and measures only the tests missing from the map, so it takes seconds on a quiet release and writes no diff at all when nothing was missing. Read the coverage line it prints before judging how long it should take — past roughly 40% of the suite missing it falls back to re-measuring everything, which takes minutes; treat a long run as a hang only when it reported few tests missing. Include the file in the commit only when it changed. `make store-test-durations-force` is **not** part of the release flow; it is for when recorded values are no longer comparable to each other because the machine or the suite changed shape. The rationale is `docs/contribute/test-duration-map.md`.- **`pipelex/migration/ledgers/*.toml` and `pipelex/migration/goldens/`** — only when the migration gate below finds an unaccounted schema change, and then written by the `add-migration` skill, never by hand.@@ -63,7 +76,7 @@ git -C <main> fetch --tags --prune origin && git -C <main> tag --list vX.Y.Z## The release commit-`pyproject.toml`, `CHANGELOG.md`, `uv.lock`, `.badges/tests.json`, `api/docs/openapi/pipelex-api.openapi.yaml`, `.test_durations` when `make store-test-durations` changed it, whatever `make agent-check` rewrote, and the `pipelex/migration/ledgers/*.toml` and `pipelex/migration/goldens/` files the `add-migration` skill wrote when it ran. By name.+`pyproject.toml`, `CHANGELOG.md`, `uv.lock`, `.badges/tests.json`, `docs/api-server/openapi/pipelex-api.openapi.yaml`, `.test_durations` when `make store-test-durations` changed it, whatever `make agent-check` rewrote, and the `pipelex/migration/ledgers/*.toml` and `pipelex/migration/goldens/` files the `add-migration` skill wrote when it ran. By name.## CI on the release pull request@@ -77,15 +90,17 @@ The checks that exist for the release:The pre-main gates a release pull request meets that a pull request to `dev` never does — they are slower, and a red here is the release stopping:-- **`lint-fresh-check.yml`** — the read-only lint suite across every supported Python version with no mypy cache, so incremental-mypy drift and version-specific breakage cannot reach `main`.-- **`tests-full-check.yml`** — the full Python matrix, sharded and balanced by `.test_durations`.+- **`lint-fresh-check.yml`** — the read-only lint suite across every supported Python version with no mypy cache, so incremental-mypy drift and version-specific breakage cannot reach `main`. Each version's leg checks the API server too.+- **`tests-full-check.yml`** — the full Python matrix, sharded and balanced by `.test_durations`, and the API server's tests on every supported version (`Tests full (api, pyX)`).- **`doc-check.yml`** — `mkdocs build --strict`, which runs unconditionally when the base is `main` rather than only when `docs/` changed.-Everything that runs on every pull request gates it too, including `lint-check.yml`'s `Lint (agent-rules)` and `Lint (config-sync)` jobs (`make check-rules` and `make check-config-sync`, neither of which `make agent-check` runs), `tests-check.yml`, `mthds-standard-check.yml`, and `dependency-review.yml`, which fails on a newly introduced dependency vulnerable at moderate severity or above and runs on pull requests to `dev` just the same. A red in `mthds-standard-check.yml` with no pinned-set change in the branch means the MTHDS standard moved, and the remedy is a dedicated change bringing the pinned natives to the standard's page — never a tweak to the release branch.+Everything that runs on every pull request gates it too, including the server's `Lint (api)`, `Tests (api)`, `Tests (packages)`, which builds both packages and checks the `pipelex-api` pin, and `Tests (api image)`, which builds and smoke-tests the image, and `lint-check.yml`'s `Lint (agent-rules)` and `Lint (config-sync)` jobs (`make check-rules` and `make check-config-sync`, neither of which `make agent-check` runs), `tests-check.yml`, `mthds-standard-check.yml`, and `dependency-review.yml`, which fails on a newly introduced dependency vulnerable at moderate severity or above and runs on pull requests to `dev` just the same. A red in `mthds-standard-check.yml` with no pinned-set change in the branch means the MTHDS standard moved, and the remedy is a dedicated change bringing the pinned natives to the standard's page — never a tweak to the release branch.## Particulars+- **The first release that ships the server explains the image's version jump.** The `pipelex/pipelex-api` image was released on its own until `0.33.2`, from the `Pipelex/pipelex-api` repository; its next tag is this release's version. `[Unreleased]` already says so in its `The Pipelex API server ships with pipelex` entry: keep that sentence when folding the entry, and say it in the pull request body. Releases after that one need no such note.+- **A server change is an entry in this changelog.** `api/CHANGELOG.md` stops at `0.33.2` and is never extended.- **The migration gate feeds back into the bump.** A breaking entry found by the cross-check above turns a patch into a minor, so the bump is not final until that gate has run.-- **The pre-release track is a different flow, not this play.** `pre-release/vX.Y.Z(a|b|rc)N` is a *base* branch that work merges into: `prerelease-version-check.yml` validates the PEP 440 form against the branch name, `publish-pypi.yml` also fires on a merge into it and marks the GitHub Release a pre-release, and `deploy-docs.yml` fires on the push to it as well — though its pre-release branch calls `make docs-deploy-specific-version`, a name with no recipe behind it (the recipe is `docs-deploy-specific-version-pre-release`), so that step succeeds and publishes nothing. A `release/vX.Y.Z` branch therefore never carries a pre-release version: `version-check.yml` fails on the mismatch with the branch name, and `guard-branches.yml` refuses a `release/vX.Y.Z(a|b|rc)N` head into `main` outright.+- **The pre-release track is a different flow, not this play.** `pre-release/vX.Y.Z(a|b|rc)N` is a *base* branch that work merges into: `prerelease-version-check.yml` validates the PEP 440 form against the branch name, `publish-pypi.yml` also fires on a merge into it, marks the GitHub Release a pre-release and publishes the server's wheel and image as well, the image under its own tag with `latest` left on the last stable release, and `deploy-docs.yml` fires on the push to it as well — though its pre-release branch calls `make docs-deploy-specific-version`, a name with no recipe behind it (the recipe is `docs-deploy-specific-version-pre-release`), so that step succeeds and publishes nothing. A `release/vX.Y.Z` branch therefore never carries a pre-release version: `version-check.yml` fails on the mismatch with the branch name, and `guard-branches.yml` refuses a `release/vX.Y.Z(a|b|rc)N` head into `main` outright.- **The changelog heading carries the `v`** — `## [vX.Y.Z] - YYYY-MM-DD`, which is exactly what `changelog-check.yml` greps for and what `publish-pypi.yml` slices the GitHub Release notes out of. No `[Unreleased]` heading is left behind; the next change re-creates one.- **`.worktreeinclude` names the gitignored files a fresh worktree needs** — `.env`, the `.pipelex/` overrides and `.pipelex-dev/test_profiles_override.toml` — and `wt add` provisions them. A gate that fails in `_pipelex--release` on a missing local config means that file is short: add it there rather than hand-copying the file every release.diff --git a/.dockerignore b/.dockerignoreindex 229678834..93ab690dd 100644--- a/.dockerignore+++ b/.dockerignore@@ -18,6 +18,7 @@# The server: its metadata, its package, and the configuration the image ships as /root/.pipelex.!api/pyproject.toml+!api/hatch_build.py!api/README.md!api/LICENSE!api/.gitignorediff --git a/.github/workflows/lint-check.yml b/.github/workflows/lint-check.ymlindex 1b0f95467..fc40d3fab 100644--- a/.github/workflows/lint-check.yml+++ b/.github/workflows/lint-check.yml@@ -234,16 +234,61 @@ jobs:run: make check-rules# ---------------------------------------------------------------------------# 9. Aggregator job — the required status check that covers every lint job,+# 9. The API server (`api/`, the `pipelex-api` workspace member) — its own+# ruff, pyright and mypy, and the drift checks of its committed OpenAPI+# artifact and of its vendored inference tree, in its own environment+# (`api/.venv`) against the pipelex of this commit. No path filter: the+# server runs the library of the same commit, so a library change can break+# it, move its wire or leave its inference tree behind, and this pull+# request is where that is found. Floor Python only, as for the library; the+# pre-main gate (`lint-fresh-check.yml`) checks every supported version.+# --------------------------------------------------------------------------+ lint-api:+ name: Lint (api)+ runs-on: ubuntu-latest++ steps:+ - uses: actions/checkout@v4++ - name: Install uv (with cache)+ uses: astral-sh/setup-uv@v7+ with:+ enable-cache: true+ cache-dependency-glob: "uv.lock"++ - name: Install the server's environment+ run: PYTHON_VERSION=3.11 make -C api install++ - name: Run ruff format merge check+ run: make -C api merge-check-ruff-format++ - name: Run ruff lint merge check+ run: make -C api merge-check-ruff-lint++ - name: Run pyright merge check+ run: make -C api merge-check-pyright++ - name: Run mypy merge check+ run: make -C api merge-check-mypy++ - name: Check the committed OpenAPI artifact for drift+ run: make -C api openapi-check++ - name: Check the vendored inference tree for drift+ run: make -C api kit-check++# --------------------------------------------------------------------------+# 10. Aggregator job — the required status check that covers every lint job,# including the ones the rulesets do not name individually-# (`Lint (hub layering)`, `Lint (rich imports)`, `Lint (agent-rules)`). It is-# not the only required lint check: the rulesets name several of the jobs-# above by name as well. Anything added to `needs` becomes blocking through here.+# (`Lint (hub layering)`, `Lint (rich imports)`, `Lint (agent-rules)`,+# `Lint (api)`). It is not the only required lint check: the rulesets name+# several of the jobs above by name as well. Anything added to `needs`+# becomes blocking through here.# --------------------------------------------------------------------------lint-all:name: Lint (all)runs-on: ubuntu-latest- needs: [lint-ruff-plxt, lint-typecheck, lint-config-sync, lint-keyword-only, lint-hub-layering, lint-rich-imports, lint-agent-rules]+ needs: [lint-ruff-plxt, lint-typecheck, lint-config-sync, lint-keyword-only, lint-hub-layering, lint-rich-imports, lint-agent-rules, lint-api]if: always()steps:@@ -255,7 +300,8 @@ jobs:[ "${{ needs.lint-keyword-only.result }}" != "success" ] || \[ "${{ needs.lint-hub-layering.result }}" != "success" ] || \[ "${{ needs.lint-rich-imports.result }}" != "success" ] || \- [ "${{ needs.lint-agent-rules.result }}" != "success" ]; then+ [ "${{ needs.lint-agent-rules.result }}" != "success" ] || \+ [ "${{ needs.lint-api.result }}" != "success" ]; thenecho "::error::At least one lint job failed."exit 1fidiff --git a/.github/workflows/lint-fresh-check.yml b/.github/workflows/lint-fresh-check.ymlindex 03b214caf..6afb812c9 100644--- a/.github/workflows/lint-fresh-check.yml+++ b/.github/workflows/lint-fresh-check.yml@@ -79,6 +79,24 @@ jobs:- name: Run mypy merge check (Python ${{ matrix.python-version }}, fresh)run: LINT_PYTHON_VERSION=${{ matrix.python-version }} make merge-check-mypy+ # The API server (`api/`), checked the same way in its own environment: `Lint (api)` in+ # lint-check.yml checks it on the floor version on every pull request, and this leg adds the+ # version this matrix row names, so the versions its classifiers claim are the ones checked.+ - name: Install the API server's environment+ run: PYTHON_VERSION=3.11 make -C api install++ - name: Run the API server's ruff merge checks+ run: make -C api merge-check-ruff-format merge-check-ruff-lint++ - name: Run the API server's pyright merge check (Python ${{ matrix.python-version }})+ run: LINT_PYTHON_VERSION=${{ matrix.python-version }} make -C api merge-check-pyright++ - name: Run the API server's mypy merge check (Python ${{ matrix.python-version }}, fresh)+ run: LINT_PYTHON_VERSION=${{ matrix.python-version }} make -C api merge-check-mypy++ - name: Run the API server's drift checks+ run: make -C api openapi-check kit-check+# --------------------------------------------------------------------------# Aggregator — single required status check for the gate.# --------------------------------------------------------------------------diff --git a/.github/workflows/publish-docker-hub.yml b/.github/workflows/publish-docker-hub.ymlnew file mode 100644index 000000000..e8934ce0c--- /dev/null+++ b/.github/workflows/publish-docker-hub.yml@@ -0,0 +1,238 @@+name: Publish the API server image 🐳 to Docker Hub++# The third artifact of a release, beside the `pipelex` and `pipelex-api` distributions: the+# `pipelex/pipelex-api` image, built from the commit the `vX.Y.Z` tag names and tagged with that+# version, which is the version of the pipelex it runs.+#+# Two ways in, one job, and both name the release by its version rather than by a commit:+#+# - `workflow_call` from `publish-pypi.yml`, as the last link of the release chain, once both+# distributions are on PyPI and the GitHub Release has tagged the release commit.+# - `workflow_dispatch`, the retry. Re-running the release run's failed jobs is the first resort and+# needs nothing else; dispatching this workflow with the version is for when the fix is a change to+# this file, which a re-run would not pick up. Dispatch it from the branch carrying that fix: the+# job builds the commit the tag names, never the head of the ref it was dispatched from. The+# Dockerfile and the `deploy-docker-hub` recipe are that commit's too, so only a fix to this file+# reaches a retry.+#+# A version already on Docker Hub is never built again, because a rebuild is a different image and a+# version tag names one image for good: a retry that finds it there only moves `latest`. `latest`+# moves to the newest stable release and nowhere else, so a pre-release, or a retry of an older+# release, leaves it where it is.++on:+ workflow_call:+ inputs:+ version:+ description: "The release's version, X.Y.Z; its tag vX.Y.Z names the commit to build."+ type: string+ required: true+ secrets:+ DOCKER_HUB_TOKEN:+ required: true+ workflow_dispatch:+ inputs:+ version:+ description: "The release whose image to publish, X.Y.Z; its tag vX.Y.Z must exist."+ type: string+ required: true++jobs:+ publish:+ name: Build, smoke-test, push and confirm the image+ runs-on: ubuntu-latest+ # One push at a time, never cancelled half-way: a second release, or a retry dispatched while the+ # release run is still pushing, waits for the first. GitHub keeps one waiting job per group, so a+ # third one queued behind them replaces the waiting one, which a retry can simply be dispatched again for.+ concurrency:+ group: publish-docker-hub+ cancel-in-progress: false++ permissions:+ contents: read++ env:+ VERSION: ${{ inputs.version }}++ steps:+ # The tag is created by `github-release` in `publish-pypi.yml`, which runs before this job on the+ # release path, and it names the commit that release shipped, so that commit is the one built.+ # Uses the commits endpoint so an annotated tag resolves to its commit, and tells "no such tag"+ # from "the API could not answer", as the guard there does.+ - name: Resolve the release commit from its tag, and whether latest moves+ id: release+ env:+ GH_TOKEN: ${{ github.token }}+ run: |+ set -euo pipefail+ if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+((a|b|rc)[0-9]+)?$ ]]; then+ echo "::error::'$VERSION' is not a release version (X.Y.Z, or X.Y.Z followed by aN, bN or rcN)."+ exit 1+ fi+ if BODY=$(gh api "repos/$GITHUB_REPOSITORY/commits/v$VERSION" 2>/dev/null); then+ RELEASE_SHA=$(jq -r '.sha' <<<"$BODY")+ else+ case "$(jq -r '.status // empty' <<<"$BODY" 2>/dev/null || true)" in+ 404|422)+ echo "::error::v$VERSION is not tagged, so there is no released commit to build. The image is published only after the GitHub Release has tagged the release; run this once it has."+ exit 1+ ;;+ *)+ REASON=$(jq -r '.message // empty' <<<"$BODY" 2>/dev/null || true)+ echo "::error::Could not determine which commit v$VERSION tags: ${REASON:-the GitHub API call returned no response body}. Refusing to build an image without knowing it is the release's."+ exit 1+ ;;+ esac+ fi+ # A pre-release (PEP 440 a, b or rc) gets its own tag only. A stable version moves `latest`+ # only while no newer stable version is tagged, so a retry of an older release cannot pull+ # `latest` back from the newest one.+ if [[ "$VERSION" =~ (a|b|rc)[0-9]+$ ]]; then+ PUSH_LATEST=false+ else+ NEWEST=$(gh api --paginate "repos/$GITHUB_REPOSITORY/git/matching-refs/tags/v" --jq '.[].ref' \+ | sed 's#^refs/tags/v##' | { grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' || true; } | sort -V | tail -n 1)+ if [ "$NEWEST" = "$VERSION" ]; then+ PUSH_LATEST=true+ else+ PUSH_LATEST=false+ echo "::notice::v$NEWEST is the newest stable release, so latest stays on it and only $VERSION is published."+ fi+ fi+ {+ echo "sha=$RELEASE_SHA"+ echo "push_latest=$PUSH_LATEST"+ } >> "$GITHUB_OUTPUT"+ echo "Publishing pipelex/pipelex-api:$VERSION from $RELEASE_SHA (latest moves: $PUSH_LATEST)"++ - uses: actions/checkout@v4+ with:+ ref: ${{ steps.release.outputs.sha }}+ persist-credentials: false++ - name: Check the tagged commit is this version+ run: |+ set -euo pipefail+ CHECKED_OUT=$(grep -m 1 '^version = ' pyproject.toml | cut -d '"' -f 2)+ if [ "$CHECKED_OUT" != "$VERSION" ]; then+ echo "::error::v$VERSION tags a commit whose pyproject.toml says version '$CHECKED_OUT'. Refusing to publish it as $VERSION."+ exit 1+ fi++ # Docker Hub's public tags API: a 200 means this version was pushed before, by an earlier attempt+ # or an earlier run, and that image stays the version's image.+ - name: Ask Docker Hub whether this version is already published+ id: hub+ run: |+ set -euo pipefail+ CODE=$(curl -sS --retry 3 --retry-delay 2 --retry-all-errors \+ -o "$RUNNER_TEMP/hub-version-tag.json" -w '%{http_code}' \+ "https://hub.docker.com/v2/namespaces/pipelex/repositories/pipelex-api/tags/$VERSION" || echo "transport-failure")+ case "$CODE" in+ 200)+ DIGEST=$(jq -r '.digest' "$RUNNER_TEMP/hub-version-tag.json")+ echo "::notice::pipelex/pipelex-api:$VERSION is already on Docker Hub ($DIGEST); it is kept, not rebuilt."+ {+ echo "published=true"+ echo "digest=$DIGEST"+ } >> "$GITHUB_OUTPUT"+ ;;+ 404)+ echo "pipelex/pipelex-api:$VERSION is not on Docker Hub yet."+ echo "published=false" >> "$GITHUB_OUTPUT"+ ;;+ *)+ echo "::error::Could not ask Docker Hub whether pipelex/pipelex-api:$VERSION exists (got '$CODE'). Refusing to push without knowing whether it would replace a published image."+ exit 1+ ;;+ esac++ # `deploy-docker-hub` builds the image from the repository root, smoke-tests it (it must boot and+ # report this version for the server and the pipelex it runs), and only then pushes it.+ - name: Build, smoke-test and push the image+ if: steps.hub.outputs.published != 'true'+ env:+ DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}+ PUSH_LATEST: ${{ steps.release.outputs.push_latest }}+ run: make -C api deploy-docker-hub PUSH_LATEST="$PUSH_LATEST"++ - name: Point latest at the image already published+ if: steps.hub.outputs.published == 'true' && steps.release.outputs.push_latest == 'true'+ env:+ DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}+ run: make -C api deploy-docker-hub-latest++ # Docker Hub's own answer, read back from its public tags API rather than from the local daemon:+ # the version tag exists for linux/amd64, and `latest`, when it moved, names the same image. Each+ # read is retried until it matches, because the tags API can lag a push by a few seconds and+ # `latest` already exists, so a read too early answers with the previous release's image.+ - name: Confirm the tags on Docker Hub+ env:+ KEPT_DIGEST: ${{ steps.hub.outputs.digest }}+ PUSH_LATEST: ${{ steps.release.outputs.push_latest }}+ run: |+ set -euo pipefail+ # hub_digest TAG [EXPECTED_DIGEST]: prints the tag's digest once Docker Hub serves it for+ # linux/amd64, and, when an expected digest is given, once it is that digest.+ hub_digest() {+ local tag="$1" expected="${2:-}" attempt code digest="" seen="nothing"+ for attempt in 1 2 3 4 5 6 7 8 9 10 11 12; do+ code=$(curl -sS -o "$RUNNER_TEMP/hub-tag.json" -w '%{http_code}' \+ "https://hub.docker.com/v2/namespaces/pipelex/repositories/pipelex-api/tags/$tag" || echo "transport-failure")+ if [ "$code" = "200" ]; then+ digest=$(jq -r '.digest' "$RUNNER_TEMP/hub-tag.json")+ seen="$digest"+ if jq -e '[.images[]?.architecture] | index("amd64")' "$RUNNER_TEMP/hub-tag.json" >/dev/null \+ && { [ -z "$expected" ] || [ "$digest" = "$expected" ]; }; then+ echo "$digest"+ return 0+ fi+ else+ seen="HTTP $code"+ fi+ echo "Docker Hub serves pipelex/pipelex-api:$tag as $seen (attempt $attempt); retrying." >&2+ sleep 10+ done+ echo "::error::Docker Hub serves pipelex/pipelex-api:$tag as $seen, not ${expected:-a linux/amd64 image}, after the push." >&2+ return 1+ }+ VERSION_DIGEST=$(hub_digest "$VERSION" "$KEPT_DIGEST")+ echo "pipelex/pipelex-api:$VERSION is $VERSION_DIGEST on Docker Hub"+ if [ "$PUSH_LATEST" = "true" ]; then+ hub_digest latest "$VERSION_DIGEST" >/dev/null+ echo "pipelex/pipelex-api:latest names the same image"+ fi+ if [ -n "$KEPT_DIGEST" ]; then+ SUMMARY="Kept \`pipelex/pipelex-api:$VERSION\` ($VERSION_DIGEST), which was already published"+ else+ SUMMARY="Published \`pipelex/pipelex-api:$VERSION\` ($VERSION_DIGEST)"+ fi+ if [ "$PUSH_LATEST" = "true" ]; then+ SUMMARY="$SUMMARY, and \`latest\` names it"+ fi+ {+ echo "$SUMMARY."+ echo "https://hub.docker.com/r/pipelex/pipelex-api"+ } >> "$GITHUB_STEP_SUMMARY"++ # The Docker Hub overview is the server's README, pushed with the image of the newest stable+ # release so it never lags behind the repository. It is cosmetic beside the image, so a failure+ # here is a warning and not a failed release: the usual cause is a token without the+ # read/write/delete scope Docker Hub requires for editing a repository's description.+ - name: Update the Docker Hub overview from the server's README+ id: overview+ if: steps.release.outputs.push_latest == 'true'+ continue-on-error: true+ uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0+ with:+ username: pipelex+ password: ${{ secrets.DOCKER_HUB_TOKEN }}+ repository: pipelex/pipelex-api+ readme-filepath: api/README.md++ - name: Report an overview that was not updated+ if: steps.release.outputs.push_latest == 'true' && steps.overview.outcome != 'success'+ run: >-+ echo "::warning::The image is published, but the Docker Hub overview was not updated from+ api/README.md. Check that DOCKER_HUB_TOKEN has the read/write/delete scope, then re-run this+ job, or paste the README into the repository's overview on Docker Hub."diff --git a/.github/workflows/publish-pypi.yml b/.github/workflows/publish-pypi.ymlindex 2c7393ee9..e0dd4428f 100644--- a/.github/workflows/publish-pypi.yml+++ b/.github/workflows/publish-pypi.yml@@ -1,4 +1,21 @@-name: Publish Python 🐍 distribution 📦 to PyPI+name: Publish the release 📦 to PyPI and Docker Hub++# One release, three artifacts under one version number: the `pipelex` distribution, the+# `pipelex-api` distribution (the API server, the `api/` workspace member) and the+# `pipelex/pipelex-api` image. The order is deliberate:+#+# build -> build-api -> publish-to-pypi -> github-release -> publish-api-to-pypi -> publish-docker-hub+#+# Both distributions are built and checked before anything is uploaded, so a server that does not+# build stops the release while the version is still free, rather than after `pipelex` X.Y.Z is on+# PyPI and tagged, when no fixed commit could ship under it. Nothing of the server ships before the+# `pipelex` wheel is on PyPI, because the `pipelex-api` wheel pins `pipelex==X.Y.Z` exactly, nor+# before `github-release`, whose tag guard is what proves the commit this run checked out is the+# release's: a retry from a ref whose head the tag does not name fails there, before a server file+# reaches PyPI, where it could never be replaced. The image goes last, built from the commit+# `vX.Y.Z` names. Each publishing job can be re-run on its own with "Re-run failed jobs"; the image+# can also be pushed again by dispatching `publish-docker-hub.yml` with the version, without this+# workflow.on:pull_request:@@ -24,6 +41,9 @@ jobs:contents: readactions: read # reading artifact lists — this run's, and the repository's+ outputs:+ version: ${{ steps.project.outputs.version }}+env:# The commit whose build this job is looking for, resolved into HEAD_SHA# below. The two triggers name it differently: on the `pull_request` path@@ -347,6 +367,9 @@ jobs:if: github.event_name == 'workflow_dispatch' || github.event.pull_request.merged == trueneeds:- build+ # The server is built and its pin checked before this irrevocable upload, so a server that+ # does not build stops the release while X.Y.Z can still ship from a fixed commit.+ - build-apiruns-on: ubuntu-latestenvironment:name: pypi@@ -385,6 +408,95 @@ jobs:# guard there is what keeps it intact on the two paths this flag opens.skip-existing: ${{ github.run_attempt > 1 || github.event_name == 'workflow_dispatch' }}+ # The server's distribution. It is built from the same checkout as `pipelex` and, unlike it, is+ # simply rebuilt on a retry: it is not attached to the GitHub Release or signed, so no recovered+ # bytes need to match, and `skip-existing` below leaves PyPI's copy alone once it is there. Which+ # commit that checkout is gets settled by `github-release`'s tag guard, which the upload waits for.+ # `hatch_build.py` writes the exact pin on `pipelex` into its metadata, and the check below refuses+ # to store a build that does not carry it, so a missing or wrong pin never reaches PyPI.+ build-api:+ name: Build the pipelex-api distribution 📦+ if: github.event_name == 'workflow_dispatch' || github.event.pull_request.merged == true+ needs:+ - build # the dispatched-ref guard runs there, and the version comes from there+ runs-on: ubuntu-latest++ permissions:+ contents: read++ steps:+ - uses: actions/checkout@v4+ with:+ persist-credentials: false+ - name: Set up Python+ uses: actions/setup-python@v5+ with:+ python-version: "3.x"+ - name: Install pypa/build+ run: >-+ python3 -m+ pip install+ build+ --user+ - name: Build a binary wheel and a source tarball of the server+ run: python3 -m build api --outdir dist-api+ - name: Verify they are this version's and pin pipelex to it exactly+ env:+ VERSION: ${{ needs.build.outputs.version }}+ run: python3 api/scripts/check_lockstep_pin.py dist-api --version "$VERSION"+ - name: Store the pipelex-api distribution packages+ uses: actions/upload-artifact@v4+ with:+ name: pipelex-api-distributions+ path: dist-api/+ # A "Re-run all jobs" rebuilds, and its build replaces the one an earlier attempt stored.+ overwrite: true++ publish-api-to-pypi:+ name: >-+ Publish the pipelex-api distribution 📦 to PyPI+ if: github.event_name == 'workflow_dispatch' || github.event.pull_request.merged == true+ needs:+ - build-api+ # After `publish-to-pypi` through it, since the wheel pins pipelex==X.Y.Z, which must be+ # installable first; and after its tag guard, which refuses a checkout the tag does not name.+ - github-release+ runs-on: ubuntu-latest+ environment:+ name: pypi+ url: https://pypi.org/p/pipelex-api+ permissions:+ id-token: write # IMPORTANT: mandatory for trusted publishing++ steps:+ - name: Download the pipelex-api dists+ uses: actions/download-artifact@v4+ with:+ name: pipelex-api-distributions+ path: dist/+ - name: Publish the pipelex-api distribution 📦 to PyPI+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2+ with:+ # The same rule as `publish-to-pypi`: only a retry may skip files PyPI already holds.+ skip-existing: ${{ github.run_attempt > 1 || github.event_name == 'workflow_dispatch' }}++ # The image, last, once both distributions are on PyPI. `publish-docker-hub.yml` builds the commit+ # the `vX.Y.Z` tag names, which `github-release` has just created.+ publish-docker-hub:+ name: Publish the API server image 🐳 to Docker Hub+ if: github.event_name == 'workflow_dispatch' || github.event.pull_request.merged == true+ needs:+ - build # the version+ - github-release+ - publish-api-to-pypi+ permissions:+ contents: read+ uses: ./.github/workflows/publish-docker-hub.yml+ with:+ version: ${{ needs.build.outputs.version }}+ secrets:+ DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}+github-release:name: >-Create GitHub Release with Changelogdiff --git a/.github/workflows/tests-check.yml b/.github/workflows/tests-check.ymlindex 22a282406..563080ba2 100644--- a/.github/workflows/tests-check.yml+++ b/.github/workflows/tests-check.yml@@ -106,20 +106,108 @@ jobs:run: make test-ts-gates# ---------------------------------------------------------------------------# 3. Aggregator job — the *single* required status check-# Guards against a silently-dropped shard: fails unless ALL 8 succeed.+# 3. The API server (`api/`, the `pipelex-api` workspace member) — its unit+# tests, in its own environment (`api/.venv`) against the pipelex of this+# commit. No path filter, for the reason `Lint (api)` gives: a library change+# can break the server. Floor Python only; the pre-main gate+# (`tests-full-check.yml`) runs every supported version.+# --------------------------------------------------------------------------+ api-tests:+ name: Tests (api)+ runs-on: ubuntu-latest+ env:+ ENV: dev+ steps:+ - uses: actions/checkout@v4++ - name: Install uv (with cache)+ uses: astral-sh/setup-uv@v7+ with:+ enable-cache: true+ cache-dependency-glob: "uv.lock"++ - name: Install the server's environment+ run: PYTHON_VERSION=3.11 make -C api install++ - name: Run the server's tests+ run: make -C api gha-tests++# --------------------------------------------------------------------------+# 4. The release's packages, built as the release builds them but published+# nowhere: the `pipelex` and `pipelex-api` wheels and sdists, the exact pin+# the `pipelex-api` metadata must carry on the `pipelex` of the same version+# (`api/scripts/check_lockstep_pin.py`, which `publish-pypi.yml` runs before+# it uploads), and the two wheels installed together into a bare environment+# with the server imported from it, as `pip install pipelex-api` does once+# both are on PyPI. The third-party versions are the lock's, so an upstream+# release cannot fail this gate; catching that is `dependency-canary.yml`'s job.+# --------------------------------------------------------------------------+ packages:+ name: Tests (packages)+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4++ - name: Install uv (with cache)+ uses: astral-sh/setup-uv@v7+ with:+ enable-cache: true+ cache-dependency-glob: "uv.lock"++ - name: Build the pipelex and pipelex-api distributions+ run: |+ uv build --package pipelex --out-dir dist+ uv build --package pipelex-api --out-dir dist++ - name: Check that the pipelex-api distributions pin pipelex exactly+ run: python3 api/scripts/check_lockstep_pin.py dist++ - name: Install the two wheels together and import the server+ run: |+ uv export --frozen --no-hashes --no-dev --no-emit-workspace --package pipelex-api -o "$RUNNER_TEMP/locked-versions.txt"+ uv venv "$RUNNER_TEMP/packages-venv" --python 3.11+ uv pip install --python "$RUNNER_TEMP/packages-venv/bin/python" --constraint "$RUNNER_TEMP/locked-versions.txt" \+ dist/pipelex-*.whl dist/pipelex_api-*.whl+ "$RUNNER_TEMP/packages-venv/bin/python" -c "import pipelex_api.main"++# --------------------------------------------------------------------------+# 5. The server's image, built from the repository root as the release builds+# it (`make -C api docker-build`) and smoke-tested (`make -C api+# docker-smoke`): it boots, answers /health, and reports this version for+# both the server and the pipelex it runs. Nothing is pushed; the push is+# `publish-docker-hub.yml`, from the release commit.+# --------------------------------------------------------------------------+ api-image:+ name: Tests (api image)+ runs-on: ubuntu-latest+ steps:+ - uses: actions/checkout@v4++ - name: Build the image+ run: make -C api docker-build++ - name: Boot the image and check the versions it reports+ run: make -C api docker-smoke++# --------------------------------------------------------------------------+# 6. Aggregator job — the *single* required status check+# Guards against a silently-dropped shard: fails unless ALL 8 succeed, and+# carries every other test job above into the required check with them.# --------------------------------------------------------------------------tests-all:name: Tests (all)runs-on: ubuntu-latest- needs: [matrix-test, ts-emission-gates]+ needs: [matrix-test, ts-emission-gates, api-tests, packages, api-image]if: always()steps:- name: Fail if any test job failedrun: |if [ "${{ needs.matrix-test.result }}" != "success" ] || \- [ "${{ needs.ts-emission-gates.result }}" != "success" ]; then+ [ "${{ needs.ts-emission-gates.result }}" != "success" ] || \+ [ "${{ needs.api-tests.result }}" != "success" ] || \+ [ "${{ needs.packages.result }}" != "success" ] || \+ [ "${{ needs.api-image.result }}" != "success" ]; thenecho "::error::At least one test job failed."exit 1fidiff --git a/.github/workflows/tests-full-check.yml b/.github/workflows/tests-full-check.ymlindex f20a7a6b0..be3d45be4 100644--- a/.github/workflows/tests-full-check.yml+++ b/.github/workflows/tests-full-check.yml@@ -69,21 +69,55 @@ jobs:echo "No hang dumps were produced."fi+# --------------------------------------------------------------------------+# The API server (`api/`) on every supported version, in its own environment+# against the pipelex of this commit. Its suite is small, so one job per+# version, unsharded. `Tests (api)` in tests-check.yml runs the floor version+# on every pull request.+# --------------------------------------------------------------------------+ api-test-full:+ name: Tests full (api, py${{ matrix.python-version }})+ runs-on: ubuntu-latest+ strategy:+ fail-fast: false+ matrix:+ python-version: ["3.11", "3.12", "3.13", "3.14"]+ env:+ ENV: dev+ steps:+ - uses: actions/checkout@v4+ with:+ ref: ${{ inputs.ref || github.sha }}++ - name: Install uv (with cache)+ uses: astral-sh/setup-uv@v7+ with:+ enable-cache: true+ cache-dependency-glob: "uv.lock"++ - name: Install the server's environment+ run: PYTHON_VERSION=${{ matrix.python-version }} make -C api install++ - name: Run the server's tests (py${{ matrix.python-version }})+ run: make -C api gha-tests+# --------------------------------------------------------------------------# Aggregator — the single required status check for the full-matrix gate.-# Green only if all 32 shards across all 4 versions pass.+# Green only if every shard of every version passes, and the API server's+# tests on every version with them.# --------------------------------------------------------------------------tests-full-all:name: Tests full (all)runs-on: ubuntu-latest- needs: [matrix-test-full]+ needs: [matrix-test-full, api-test-full]if: always()steps:- name: Fail unless every version and shard passedrun: |- if [ "${{ needs.matrix-test-full.result }}" != "success" ]; then- echo "::error::At least one full-matrix shard failed."+ if [ "${{ needs.matrix-test-full.result }}" != "success" ] || \+ [ "${{ needs.api-test-full.result }}" != "success" ]; then+ echo "::error::At least one full-matrix job failed."exit 1fi- echo "All full-matrix shards passed."+ echo "All full-matrix jobs passed."diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 3107dc30b..ab84a8520 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,7 +8,7 @@- **Document engines are models**: the engine that prints a `PipeDocGen` step is a model of a new `doc_gen` family, which a step names with `model = "pipelex-pdf"` or leaves to the default for its format and source in the new deck file `5_doc_gen_deck.toml`, and whose worker a plugin registers with `add_inference_backend(family=InferenceFamily.DOC_GEN, …)`, declaring its model with `add_internal_model` and its deck defaults with `add_doc_gen_default`. A step whose engine is not in the deck, does not print its format from its source, or has no installed worker is refused when the method loads, so a runtime that disables the built-in `reportlab` plugin in `[runtime.plugins] disabled` refuses every `PipeDocGen` step. An engine receives a `RenderJob` of plain data, reads the files its document names through `RenderResources` under the run's read scope, and may check a step's template file at the dry run. `pipelex update` now also refreshes `backends/internal.toml`, backing up a locally edited copy first, so run it to get the deck file and the built-in `reportlab-pdf` engine.- **The `markdown` template filter**: an HTML template turns Markdown held in a text field into HTML with `{{ order.notes | markdown }}`. Raw HTML in the source is shown as text, and only URLs with a scheme become links.- **The `Markdown` native concept**: `Markdown` refines `Text` and holds a text written in Markdown, which a `PipeLLM` with `output = "Markdown"` writes as free text and a `Markdown` input takes as a string or as `{"concept": "Markdown", "content": "…"}`; Pipelex defines it ahead of the MTHDS standard. A `Markdown` value is accepted wherever a `Text` is, while a `Text`, or any concept refining `Text`, is refused where a `Markdown` is required. `pipelex run --save-main-stuff` saves it verbatim as `main_stuff.md`, an HTML template prints `{{ report }}` and `$report` as HTML with raw HTML escaped, and `Stuff.as_markdown` and `PipeOutput.main_stuff_as_markdown` read it from Python.-- **The Pipelex API server ships from this repository**: the FastAPI runner published as the `pipelex/pipelex-api` image now lives in the `api/` directory, as the `pipelex-api` member of a uv workspace, and it runs the `pipelex` of the same commit instead of a pinned release. It is released together with `pipelex`, under the same version, so the image tag that follows `0.33.2` is a `pipelex` version; the image keeps its name, its port and its `/root/.pipelex` configuration mount, and `Pipelex/pipelex-api` keeps the server's earlier history.+- **The Pipelex API server ships with `pipelex`**: the FastAPI runner now lives in the `api/` directory, as the `pipelex-api` member of a uv workspace, and runs the `pipelex` of the same commit instead of a pinned release. Every release publishes it under `pipelex`'s version, as the new `pipelex-api` package on PyPI, which pins `pipelex` to that same version, and as the `pipelex/pipelex-api` image on Docker Hub, whose tag therefore jumps from `0.33.2`, the server's last release on its own, to this version. The image keeps its name, its port and its `/root/.pipelex` configuration mount, and `Pipelex/pipelex-api` keeps the server's earlier history.- **`x-pipelex-metadata` header on Pipelex Manifold calls**: every inference request the Manifold backend makes (chat completions, responses, image generation, extraction, search, and Claude over the Anthropic SDK) carries one JSON header holding the run's `extras` as they are, plus the run's `user_id` and `pipeline_run_id` and the step's `pipe_run_id`, `pipe_code` and `content_generation_job_id` when set, so the service can attribute each call's spend and log it. On a key collision the run's own id wins over the extras entry, and no other backend receives the header.### Changed@@ -17,6 +17,7 @@- **A string under a concept refining `Text` keeps the concept's class**: a bare string or a `{"concept": …, "content": "…"}` envelope for such a concept now holds the concept's own content class where it held a plain `TextContent`, as the `YesNo`, `Date` and `Time` envelopes already did.- **`markdown-it-py` replaces `markdown` in the dependencies**: the `markdown` package, which pipelex no longer imported, is dropped, and `markdown-it-py` with its `linkify` extra renders the Markdown of `PipeDocGen`, the `markdown` filter and the `Markdown` concept.- **The API server's import package is `pipelex_api` (Breaking)**: the server's package `api` is renamed `pipelex_api`, so an application wrapping the server imports `pipelex_api.main` and serves `pipelex_api.main:app`, and the `logger` field of the server's log records reads `pipelex_api.…`.+- **The API server's documentation is part of docs.pipelex.com**: its pages are the API Server section of the Pipelex documentation, versioned with each release, in place of the separate site at `pipelex.github.io/pipelex-api`, and the committed OpenAPI artifact is published beside them as `api-server/openapi/pipelex-api.openapi.yaml`.- **The API image is built from the `pipelex` repository**: `docker build -f api/Dockerfile .` at the repository root installs `pipelex` and `pipelex-api` from that commit, with the dependency versions of the repository's lock, which moves some provider SDKs to the versions `pipelex` is tested with. The container starts `uvicorn` from its virtual environment rather than through `uv run`, and carries no `uv`.### Fixeddiff --git a/api/.gitignore b/api/.gitignoreindex 7829d8470..45532530a 100644--- a/api/.gitignore+++ b/api/.gitignore@@ -9,9 +9,6 @@ __pycache__/dist/*.egg-info/-# mkdocs-site/-# Unit test / coverage reportshtmlcov/.pytest_cache/diff --git a/api/CHANGELOG.md b/api/CHANGELOG.mdindex c5a194758..03deba178 100644--- a/api/CHANGELOG.md+++ b/api/CHANGELOG.md@@ -58,7 +58,7 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipe- **Runs read only under their read scope (Breaking)**: `POST /v1/execute` and `POST /v1/start` take a `read_scope`, the prefix every `pipelex-storage://` key a run reads must lie under, and a scoped run reads no local path. Omitted, it is the caller's own id on a deployment that identifies callers and unscoped on a single-tenant one, so a multi-tenant host must send it; a malformed read scope, or a storage scope outside it, is a `422` `InvalidReadScope`. A read outside the scope is refused when the run makes it, as `UriReadRefusedError`: a `422` on `/execute`, and on `/start` a synchronous `422` for an input or a run that ends `FAILED` for a URL a pipe reaches during the run.- **A bundle declaring Python structure classes is refused (Breaking)**: on a sandbox-hosted deployment, a `files` or `bundle_b64` bundle whose Python declares a `StructuredContent` subclass now answers `403` `MethodStructuresRefusedError` naming each file and class, as a fetched `method_ref` package already did, and none of its modules is imported. Declare the types as MTHDS concepts instead; the unedited module `pipelex build structures` writes is accepted.- **Every template renders in Jinja's sandbox (Breaking)**: a prompt, a compose or construct template, an image or search prompt, or a condition expression may read data and call methods of plain values only. One that reaches for a Python internal, calls a method of a Pipelex or pydantic object, or reads a name starting with an underscore other than an input's `_stuff_name`, `_content_class`, `_concept_code` and `_stuff_code` fails with `Jinja2TemplateSecurityError`, and validation refuses the visible cases as `template_private_name`. `_content` and `stuff` are no longer readable on an input, and a PipeCompose construct `from` path or a `list_to_dict_keyed_by` name with a segment starting with an underscore is refused at validation.-- **Fetching a URL refuses private destinations (Breaking)**: every download of a URL a value carries, such as a document to extract or a prompt image for a provider that does not take URLs, now refuses a host that is `localhost`, a cloud metadata alias, or resolves to a private, loopback, link-local or metadata address, on every redirect hop, and fails the pipe with `SsrfBlockedError` naming the host. The guarded fetch ignores `HTTP_PROXY` and `HTTPS_PROXY`. The image keeps the guard on; a self-hosted deployment reading documents from an intranet host, or reaching the internet only through a proxy, sets `[runtime.network] is_fetch_ssrf_guard_enabled = false`, as [Configuration](docs/configuration.md) describes.+- **Fetching a URL refuses private destinations (Breaking)**: every download of a URL a value carries, such as a document to extract or a prompt image for a provider that does not take URLs, now refuses a host that is `localhost`, a cloud metadata alias, or resolves to a private, loopback, link-local or metadata address, on every redirect hop, and fails the pipe with `SsrfBlockedError` naming the host. The guarded fetch ignores `HTTP_PROXY` and `HTTPS_PROXY`. The image keeps the guard on; a self-hosted deployment reading documents from an intranet host, or reaching the internet only through a proxy, sets `[runtime.network] is_fetch_ssrf_guard_enabled = false`, as [Configuration](https://docs.pipelex.com/latest/api-server/configuration/) describes.## [v0.31.0] - 2026-09-28@@ -372,7 +372,7 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipe### Fixed-- **A source-less run request no longer slips through on an unknown key.** `RunRequest.validate_request` used to **waive** the "at least one of `pipe_code` / `mthds_contents` / method bundle" precondition whenever the body carried any key outside the declared fields, on the theory that an extension might be the method selector. That theory is obsolete under the layered extension policy written down in the Pipelex workspace spec `docs/specs/pipelex-platform-api.md`: an extension-borne method selector is resolved by the layer that owns it *before* the request reaches this server, so a body arriving here with no run source is an error whatever else it carries. The waiver was also already dead on every live path — the run routes build the model through `RunRequest.from_body`, which copies the declared fields only, so unknown keys were stripped before the validator ever saw them. Removing it is therefore behaviour-neutral, and the substantive change is the message described above. A body that does have a real run source still accepts unknown extension keys untouched: the model stays extension-open.+- **A source-less run request no longer slips through on an unknown key.** `RunRequest.validate_request` used to **waive** the "at least one of `pipe_code` / `mthds_contents` / method bundle" precondition whenever the body carried any key outside the declared fields, on the theory that an extension might be the method selector. That theory is obsolete under the layered extension policy the Pipelex platform API follows: an extension-borne method selector is resolved by the layer that owns it *before* the request reaches this server, so a body arriving here with no run source is an error whatever else it carries. The waiver was also already dead on every live path — the run routes build the model through `RunRequest.from_body`, which copies the declared fields only, so unknown keys were stripped before the validator ever saw them. Removing it is therefore behaviour-neutral, and the substantive change is the message described above. A body that does have a real run source still accepts unknown extension keys untouched: the model stays extension-open.## [v0.16.0] - 2026-08-20@@ -519,7 +519,7 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipe## [v0.10.0] - 2026-07-19### Changed-- **Breaking: `/execute` now returns `pipe_output.tokens_usages` as client wire records (`TokensUsageRecord`), not dumps of pipelex's internal usage models.** The route applies pipelex's `apply_tokens_usage_wire_shape` to the response dump, so each record is flat and closed: `model_type`, model name/id, `pipe_code`, `job_category`, `unit_job_id`, `nb_tokens_by_category`, computed USD `cost` (`null` when the model has no rate table), ISO `started_at`/`completed_at`. Runtime internals (`job_metadata` and its contexts/ids, `unit_costs`) no longer cross the wire. Null semantics unchanged (`null` = usage assembly off, `[]` = no inference). Documented in [Pipe Run](docs/pipe-run.md).+- **Breaking: `/execute` now returns `pipe_output.tokens_usages` as client wire records (`TokensUsageRecord`), not dumps of pipelex's internal usage models.** The route applies pipelex's `apply_tokens_usage_wire_shape` to the response dump, so each record is flat and closed: `model_type`, model name/id, `pipe_code`, `job_category`, `unit_job_id`, `nb_tokens_by_category`, computed USD `cost` (`null` when the model has no rate table), ISO `started_at`/`completed_at`. Runtime internals (`job_metadata` and its contexts/ids, `unit_costs`) no longer cross the wire. Null semantics unchanged (`null` = usage assembly off, `[]` = no inference). Documented in [Pipe Run](https://docs.pipelex.com/latest/api-server/pipe-run/).- **The OpenAPI artifact now publishes the wire shape it actually emits.** `/execute` returns a `JSONResponse` built from the trimmed dump, so FastAPI never serializes through `response_model` — the declared 200 body was still the internal usage union (`LLMTokensUsage | ImgGenTokensUsage | …` with required `job_metadata` and `unit_costs`), and a client generated from the artifact would have rejected every real response carrying usage data. The route now declares `PipelexApiExecuteResponse` (the run result whose `pipe_output.tokens_usages` is `TokensUsageRecord[]`), so the published schema matches the body. The internal usage models leave the artifact entirely; `tests/unit/test_openapi_contract.py` pins the published record alongside the runtime guard, so the two fail together if either drifts again.- **Dependencies:** Bumped `pipelex` from `0.39.1` to `0.40.0`. This release carries `apply_tokens_usage_wire_shape` and the `TokensUsageRecord` wire shape the `/execute` change above depends on. (It supersedes the interim `0.39.2` bump — which unblocked the hosted runner resolving against a git-sourced `dev` — with a released PyPI pin.)@@ -533,16 +533,16 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipe### Added- **`POST /v1/resolve` (MTHDS Protocol resolution capability):** resolve a library closure into its **normalized library crate** over HTTP. Request selects the closure with inline `files[]` (each `{content, source?}`) XOR a `method_ref`; a produced verdict is a `200` discriminated on `is_valid` — the canonical JSON crate (qualified refs, flattened refinement, materialized natives, fingerprint) on the valid arm, the structured `validation_errors[]` on the invalid arm. `method_ref` is accepted by the envelope but answers `501` (`MethodRefNotSupported`) until server-side method-registry resolution exists.- **`POST /v1/codegen` (MTHDS Protocol type-projection capability):** project the crate into typed artifacts with the two explicit axes — `kind` (`types`) × `target` (`ts-zod`, `python-pydantic`, and the Pipelex-extension `python-structures`). The valid arm carries the **stamped** artifact set plus its `codegen.lock`: a client that writes them verbatim reproduces a local `pipelex codegen types` run byte-for-byte and passes the offline `pipelex codegen check`. An unknown `kind`/`target` (or a `pipe_ref` on the concept-set-wide `types`) is a request-shape `422` problem+json. There is deliberately no server-side check route (the drift check is offline by design), and no `inputs` kind: served-kind membership follows the stamp/lock trust chain, and an inputs template is a user-editable scaffold that is never stamped or locked — it rides `POST /v1/build/inputs` instead.-- **Docs:** new [Resolve & Codegen](docs/codegen.md) page; [Pipe Builder](docs/pipe-builder.md) rewritten to the new envelopes.+- **Docs:** new [Resolve & Codegen](https://docs.pipelex.com/latest/api-server/codegen/) page; [Pipe Builder](https://docs.pipelex.com/latest/api-server/pipe-builder/) rewritten to the new envelopes.- **`postman-bundle` skill (renamed from `postman-run-bundle`) now covers the whole bundle surface:** besides `/v1/execute`, `/v1/start`, and `/v1/validate`, the skill and its `build_postman_query.py` script target `/v1/resolve`, `/v1/codegen` (with `--target ts-zod|python-pydantic|python-structures`), and `/v1/build/{inputs,output,runner}` (with `--output-format` for `build/output`, and `--inputs-format json|toml` + `--explicit` for `build/inputs`). New make targets `bundle-resolve` and `bundle-codegen`; the build routes ride the generic targets via `ENDPOINT=`. Postman pushes now land under a `Bundles/` top folder (was `Run Bundle/`) and merge by request name instead of replacing the whole per-bundle subfolder, so requests for different endpoints coexist.-- **`suggested_fix` on validation errors (from pipelex codegen):** a `validation_errors[]` item may now carry a structured, deterministic repair the runtime's fix planner derived from the *typed* error data (never by parsing a message string) — `fix_code` (kebab-case rule id), `description`, `safety` (`safe` | `unsafe`), an optional `source`, and `ops[]`: semantic TOML patch operations (`set_key`, `ensure_table`, `delete_key`, `delete_table`, `rename_table_key`) addressed by `table_path`. The ops are the machine contract; any rendered diff is presentation. Optional and additive — a non-fixable item omits the field and a client that ignores it is unaffected. It rides every surface that carries the items: the `200` invalid arms of `/v1/validate`, `/v1/resolve`, `/v1/codegen`, and `/v1/build/*`, and the RFC 7807 problem documents. Documented in [Error Responses → Suggested fixes](docs/error-responses.md#suggested-fixes).+- **`suggested_fix` on validation errors (from pipelex codegen):** a `validation_errors[]` item may now carry a structured, deterministic repair the runtime's fix planner derived from the *typed* error data (never by parsing a message string) — `fix_code` (kebab-case rule id), `description`, `safety` (`safe` | `unsafe`), an optional `source`, and `ops[]`: semantic TOML patch operations (`set_key`, `ensure_table`, `delete_key`, `delete_table`, `rename_table_key`) addressed by `table_path`. The ops are the machine contract; any rendered diff is presentation. Optional and additive — a non-fixable item omits the field and a client that ignores it is unaffected. It rides every surface that carries the items: the `200` invalid arms of `/v1/validate`, `/v1/resolve`, `/v1/codegen`, and `/v1/build/*`, and the RFC 7807 problem documents. Documented in [Error Responses → Suggested fixes](https://docs.pipelex.com/latest/api-server/error-responses/#suggested-fixes).### Changed- **`pipelex` is pinned to a git rev while `codegen` is unreleased.** The dependency source moved from a local editable path (`../pipelex`) to a git rev on the `pipelex` repo. The editable path only ever resolved on a workspace checkout — it fails `uv sync` anywhere else, CI included. The pinned rev carries the unreleased codegen engine this API rides; it reverts to a plain `==<version>` PyPI pin once a `pipelex` release ships it.- **The OpenAPI artifact now documents the real error contract.** Every failure this server emits is an RFC 7807 `application/problem+json` problem document, but the published artifact said otherwise: it advertised FastAPI's default `HTTPValidationError` on `application/json` for the 422 of nearly every route, `/v1/execute` and `/v1/start` documented no failure at all, and no route documented 401/403/409/413/429/500/501. Now every `/v1` operation documents `401`/`413`/`422`/`500` as a typed `ProblemDocument` under `application/problem+json`, plus the statuses it alone can produce — `/execute`: `403`, `429` (with `Retry-After`); `/start`: `400`, `403`, `409`, `501`; `/validate`: `403`; `/resolve` + `/codegen`: `501`. `GET /v1/version` stays public and 401-free. `HTTPValidationError` is gone from the artifact entirely.- **Breaking — `/v1/resolve` and `/v1/codegen` are no longer tagged `x-mthds-protocol`.** They are **Pipelex API extensions**, not MTHDS Protocol operations: the protocol is, and stays, the five routes `execute` / `start` / `validate` / `models` / `version` that the standard's own normative OpenAPI defines. The flag is how a conformance suite or a third-party runner extracts the portable subset of the artifact, so tagging a Pipelex route misrepresented the standard. The *crate* `/v1/resolve` emits remains standard-owned (the MTHDS Library Crate Format) with brand-neutral wire fields — only the routes' protocol membership changed, and no request or response shape moved. `tests/unit/test_openapi_contract.py` now pins the tagged set exactly, in both directions.-- **Docs:** [Error Responses](docs/error-responses.md) gains a *Suggested fixes* section and the missing `missing_pipe_code` field, and its status-code list now covers the `400` (`StartRequiresAsyncOrchestration`), the `orchestration_mode` `403`, and the `method_ref` `501` (the `501` bullet also named a stale `/pipeline/start` path). The [Overview](docs/index.md) endpoint catalog gains the *Resolve & Codegen* section and `GET /`, and corrects the uploader auth requirement — `/v1/upload` and `/v1/resolve-storage-url` need an authenticated **user identity**, which `AUTH_MODE=api_key` does not establish (the key is shared, not per-caller).+- **Docs:** [Error Responses](https://docs.pipelex.com/latest/api-server/error-responses/) gains a *Suggested fixes* section and the missing `missing_pipe_code` field, and its status-code list now covers the `400` (`StartRequiresAsyncOrchestration`), the `orchestration_mode` `403`, and the `method_ref` `501` (the `501` bullet also named a stale `/pipeline/start` path). The [Overview](https://docs.pipelex.com/latest/api-server/) endpoint catalog gains the *Resolve & Codegen* section and `GET /`, and corrects the uploader auth requirement — `/v1/upload` and `/v1/resolve-storage-url` need an authenticated **user identity**, which `AUTH_MODE=api_key` does not establish (the key is shared, not per-caller).- **Breaking — `/v1/build/{inputs,output,runner}` migrated onto the `/validate` verdict discipline:** a produced verdict is now always a `200` discriminated on `is_valid`. `build/inputs` and `build/output` wrap their previous bare JSON payload in the valid arm (`inputs` / `output` fields); an invalid bundle — including a failed dry-run of the requested pipe — is now a `200` `is_valid: false` with the structured `validation_errors[]` instead of a `422`. `build/runner` drops the retired `success` bool.- **Breaking — `/v1/build/runner` rides the codegen types projection (D9):** the generated script spells its imports, example inputs, and output cast with the **emitted** class names, and the valid arm now also carries the `structures` projection it imports from (stamped `structures.py` + `codegen.lock`, written into a `structures/` directory beside the script) — matching what a local `pipelex build runner` scaffolds. A requested pipe recorded SKIPPED by the sweep (cross-package dependency absent from the request) remains a request-shape `422`.- **Breaking — `/v1/build/*` now ride the same request envelope as `/v1/resolve` and `/v1/codegen`:** the closure selector is inline `files[]` (each `{content, source?}`) XOR a `method_ref` (`501` until the method registry lands), replacing the bare `mthds_contents[]` strings. The per-file `source` label means build diagnostics can finally name the owning file, and there is no second breaking migration waiting when `method_ref` resolution arrives. The pipe selector becomes **`pipe_ref`** — the qualified `domain.pipe_code`, and now **optional**: omitted, it defaults to the closure's declared `main_pipe`, exactly like `pipelex codegen inputs --pipe`. A closure declaring *no* `main_pipe`, or *several* across domains, cannot be defaulted, so an omitted `pipe_ref` is a `422` there. Every valid arm echoes the resolved `pipe_ref` plus the submitted `requested_pipe_ref` (absent when defaulted). The echoed `pipe_ref` is always **qualified**, read back off the resolved pipe rather than off the request — a bare code still resolves through the engine's cross-domain fallback, and the caller is told the qualified ref it landed on. The old bare-`mthds_contents` envelope now survives on `/v1/validate` alone, which keeps it because changing a protocol route's envelope is the MTHDS spec's call, not this server's.@@ -552,7 +552,7 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipe### Fixed- **`/v1/build/runner` could read the wrong pipe's output multiplicity.** A blueprint's pipe map is keyed by *bare* code, and the multiplicity lookup scanned by bare code alone. In a closure where two domains declare a pipe with the same name, it returned whichever blueprint came first — so a list-output pipe could be handed a scalar runner script (`main_stuff_as` instead of `main_stuff_as_items`), and vice versa. The lookup now matches the owning domain too.- **`/v1/build/output` with `format=python` was a hard `500`.** The route parsed *every* format with `json.loads` and typed the field `dict`, so the Python-source representation crashed on arrival. The representation now rides the field its format names, mirroring `/build/inputs`: `output` (a parsed object) for `schema`/`json`, `output_python` (source text) for `python`. Separately, a pipe whose `native.Anything` output has no determinable shape (which `render_output` reports with a bare `ValueError`) is now an honest request-shape `422` rather than an unhandled `500`.-- **Docs — [Error Responses](docs/error-responses.md) no longer contradicts the 200-verdict discipline.** Two sections still said an invalid bundle rides a `422` on the build routes, and the 422 example showed `POST /v1/validate` producing one; both now correctly attribute the `ValidateBundleError` 422 to the run routes (`/execute`, `/start`) alone, with the diagnostic routes answering `200` `is_valid: false`.+- **Docs — [Error Responses](https://docs.pipelex.com/latest/api-server/error-responses/) no longer contradicts the 200-verdict discipline.** Two sections still said an invalid bundle rides a `422` on the build routes, and the 422 example showed `POST /v1/validate` producing one; both now correctly attribute the `ValidateBundleError` 422 to the run routes (`/execute`, `/start`) alone, with the diagnostic routes answering `200` `is_valid: false`.## [v0.8.0] - 2026-07-06@@ -580,7 +580,7 @@ This changelog ends at v0.33.2, the server's last release from the `Pipelex/pipeOrchestrator-agnostic base. `pipelex-api` no longer hard-wires Temporal: the published base names **no** orchestrator and runs every pipeline in-process, while distributed execution (Temporal, Mistral Workflows, …) becomes a deployment *flavor* = base image + exactly one orchestrator plugin + that plugin's activation. `api/` imports no `pipelex.temporal` / `temporalio`.### Added- - **`orchestration_mode` deployment config:** a top-level run's backend is read from a new packaged `api.toml` (`orchestration_mode`, default `direct`; `allow_request_orchestration_mode_override`, default `false`), env-layered like the Pipelex config (`api_{env}.toml` / `api_override.toml`). `orchestration_mode` is an **open string token** (core owns `direct`; each orchestrator plugin owns its own, e.g. `temporal`); an unregistered token fails loud at dispatch. `POST /v1/execute`, `POST /v1/start`, and `POST /v1/validate` may each carry a per-request `orchestration_mode` override, honored only when the deployment opts in — otherwise refused with a `403` (`OrchestrationModeOverrideForbidden`). The **delivery** axis (blocking vs fire-and-forget) is endpoint-intrinsic, never configured or requestable. See [Configuration → Orchestration mode](docs/configuration.md).+ - **`orchestration_mode` deployment config:** a top-level run's backend is read from a new packaged `api.toml` (`orchestration_mode`, default `direct`; `allow_request_orchestration_mode_override`, default `false`), env-layered like the Pipelex config (`api_{env}.toml` / `api_override.toml`). `orchestration_mode` is an **open string token** (core owns `direct`; each orchestrator plugin owns its own, e.g. `temporal`); an unregistered token fails loud at dispatch. `POST /v1/execute`, `POST /v1/start`, and `POST /v1/validate` may each carry a per-request `orchestration_mode` override, honored only when the deployment opts in — otherwise refused with a `403` (`OrchestrationModeOverrideForbidden`). The **delivery** axis (blocking vs fire-and-forget) is endpoint-intrinsic, never configured or requestable. See [Configuration → Orchestration mode](https://docs.pipelex.com/latest/api-server/configuration/).- **Orchestrator HTTP-error mappers:** the app discovers each installed orchestrator plugin's transport-fault mapper (via the plugin SPI `add_http_error_mapper`) at construction and registers one RFC 7807 handler per mapped exception type — so a plugin's transport fault renders correctly while the base imports no orchestrator SDK.- **`make docker-run-hub` target:** pull and run the published `pipelex/pipelex-api` Docker Hub image without a local checkout or build, using the same `.env` contract as `make docker-run`. Pin a tag with `HUB_TAG=<tag>` (defaults to `latest`).diff --git a/api/CLAUDE.md b/api/CLAUDE.mdindex 90d0bb78d..34aba46a6 100644--- a/api/CLAUDE.md+++ b/api/CLAUDE.md@@ -6,11 +6,14 @@ Pipelex API is the official FastAPI REST server for [Pipelex](https://github.comThis directory is the `api/` member of the pipelex repository's uv workspace: the distribution `pipelex-api`, the import package `pipelex_api`, and the image `pipelex/pipelex-api`. It moved here from the `Pipelex/pipelex-api` repository at that repository's last release, v0.33.2, which remains the place to read its history; `git blame` here starts at the import. Its own `CHANGELOG.md` stops at 0.33.2, and a change to the server is now an entry in pipelex's `CHANGELOG.md`, at the repository root.-- **It runs the pipelex of the same commit.** `pyproject.toml` takes `pipelex` from the workspace (`[tool.uv.sources] pipelex = { workspace = true }`), and the repository keeps one `uv.lock`, at its root. There is no pin to bump and no git source to move: a library change reaches the server in the change that makes it, and the server's gate, which the root `make agent-check` and `make agent-test` run, fails that change when it breaks the server. pipelex's pull-request workflows do not run that gate, so run it before pushing. The dependency names no version, since uv ignores a specifier on a workspace source.-- **It has the library's version.** `pyproject.toml` declares the version dynamic and hatch reads it from the root `pyproject.toml` (`[tool.hatch.version]`), so a release bumps one file and `/v1/version`, the OpenAPI artifact and the image tag (`make deploy-docker-hub`) all carry the library's number. The published wheel's exact pin on `pipelex` is still to be added with the release chain that publishes it.+- **It runs the pipelex of the same commit.** `pyproject.toml` takes `pipelex` from the workspace (`[tool.uv.sources] pipelex = { workspace = true }`), and the repository keeps one `uv.lock`, at its root. There is no pin to bump and no git source to move: a library change reaches the server in the change that makes it, and the server's gate fails that change when it breaks the server. The root `make agent-check` and `make agent-test` run that gate, and so does every pull request, with no path filter: `Lint (api)`, `Tests (api)`, `Tests (packages)` and `Tests (api image)` in pipelex's workflows.+- **It has the library's version.** `pyproject.toml` declares the version dynamic and hatch reads it from the root `pyproject.toml` (`[tool.hatch.version]`), so a release bumps one file and `/v1/version`, the OpenAPI artifact and the image tag (`make deploy-docker-hub`) all carry the library's number.+- **Its dependencies are written in the metadata hook's table.** `dependencies` is dynamic: the server's requirements live in `[tool.hatch.metadata.hooks.custom]` of `pyproject.toml`, and `hatch_build.py` writes them into the metadata, pinning each `lockstep-dependencies` entry (pipelex with the server's extras) to `==` the release's version. uv ignores that pin on the workspace source, so it changes nothing here; in a published wheel it makes `pip install pipelex-api` take exactly the pipelex it was released with. Add or change a server dependency in that table by hand, then `make lock` at the root: `uv add` would write a static `[project] dependencies` list, which hatch refuses beside the dynamic one. `scripts/check_lockstep_pin.py` (`make build check-lockstep-pin`) checks the pin in a build, and the release workflow runs it before uploading.- **It has its own environment.** Every uv command in the Makefile targets `api/.venv` (`UV_PROJECT_ENVIRONMENT`), where `uv sync` installs this member with its extras and pipelex, in editable mode, with the server's extras and without `cli`. The targets that need the environment provision it, so a fresh worktree needs nothing run by hand first.- **Its gate runs from the root.** The root `make agent-check` ends with `make -C api agent-check` and the root `make agent-test` with `make -C api agent-test`; run them here directly while working on the server. Upgrading dependencies is done for the whole workspace, at the root.-- **The image is built from the repository root** as its context, `docker build -f api/Dockerfile .`, which is what `make docker-build` runs, so the image installs the pipelex of the same commit. The root `.dockerignore` keeps that context to the files the build reads.+- **The image is built from the repository root** as its context, `docker build -f api/Dockerfile .`, which is what `make docker-build` runs, so the image installs the pipelex of the same commit. The root `.dockerignore` keeps that context to the files the build reads. `make docker-smoke` boots the image and checks that `/v1/version` reports this version for both the server and its pipelex.+- **A pipelex release ships it.** The root's `.github/workflows/publish-pypi.yml` publishes the `pipelex-api` wheel and sdist once the `pipelex` wheel is on PyPI and the release is tagged, then calls `publish-docker-hub.yml` to push `pipelex/pipelex-api:X.Y.Z` from the commit the tag names through `make deploy-docker-hub`, which smoke-tests before it pushes. `latest` moves only to the newest stable release, and a version already on Docker Hub is never rebuilt. A failed push is retried by re-running the failed job, or by dispatching `publish-docker-hub.yml` with the version. `docs/contribute/api-server.md` at the root describes the whole chain.+- **Its documentation is part of the pipelex docs site**: the pages are `docs/api-server/` at the repository root, served by the root `make docs` and checked by `make docs-check`, and published on docs.pipelex.com with each release. The committed OpenAPI artifact sits among them, at `docs/api-server/openapi/pipelex-api.openapi.yaml`.## Project Structure@@ -37,11 +40,14 @@ pipelex_api/tests/unit/ # Unit testsscripts/- export_openapi.py # Writes or drift-checks the committed OpenAPI artifact- sync_vendored_kit.py # Writes or drift-checks the vendored .pipelex/inference/ tree+ export_openapi.py # Writes or drift-checks the committed OpenAPI artifact+ sync_vendored_kit.py # Writes or drift-checks the vendored .pipelex/inference/ tree+ check_lockstep_pin.py # Checks that a built wheel and sdist pin pipelex to the release's version+ check_served_version.py # Checks the versions a running server reports (make docker-smoke)+hatch_build.py # The metadata hook that writes the dependencies and the lockstep pin```-This server is the reference implementation of the [MTHDS Protocol](https://mthds.ai): `POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version` under the `/v1` base path, tagged `x-mthds-protocol: true` in the committed OpenAPI artifact (`docs/openapi/pipelex-api.openapi.yaml`, regenerated via `make openapi-export`, drift-checked via `make openapi-check`). Contract nesting: MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex hosted API.+This server is the reference implementation of the [MTHDS Protocol](https://mthds.ai): `POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version` under the `/v1` base path, tagged `x-mthds-protocol: true` in the committed OpenAPI artifact (`docs/api-server/openapi/pipelex-api.openapi.yaml` at the repository root, regenerated via `make openapi-export`, drift-checked via `make openapi-check`). Contract nesting: MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex hosted API.**Only those five operations carry `x-mthds-protocol`** — the flag is how a conformance suite or a third-party runner extracts the portable subset of the artifact, so tagging a Pipelex route would misrepresent the standard. Everything else this server serves is a Pipelex API extension: `/resolve` + `/codegen`, `/pipe-io`, `/build/*`, and `/lint` + `/format`. Watch `/resolve` and `/codegen` in particular: they *look* protocol-shaped (they speak the `/validate` verdict discipline, and the crate `/resolve` emits is genuinely standard-owned — the MTHDS Library Crate Format, so its wire fields stay brand-neutral), but the routes are ours and the standard specifies no type projection at all. `tests/unit/test_openapi_contract.py` pins the tagged set exactly, in both directions.@@ -69,7 +75,10 @@ make tp # Run unit tests with prints visiblemake test # Run unit tests (sequential)make gha-tests # Tests for GitHub Actions (no inference)make li # lock + install+make build # Build the pipelex-api wheel and sdist into dist/+make check-lockstep-pin # Check that the built distributions pin pipelex to this versionmake docker-build # Build the image from local source, with the repository root as the build context+make docker-smoke # Boot the built image and check /health and the versions /v1/version reportsmake docker-run # Build + run in Docker on http://localhost:8081 (foreground)make docker-run-hub # Pull + run the published Docker Hub image (no local build); HUB_TAG=<tag> to pinmake docker-stop # Force-stop the Docker container@@ -184,12 +193,12 @@ async def execute(## Error Responses-Every error is rendered as RFC 7807 `application/problem+json` by the global handlers in `pipelex_api/exception_handlers.py`. **Route handlers do not shape errors themselves** — they call into pipelex and let exceptions propagate. The wire contract is documented at `docs/error-responses.md`.+Every error is rendered as RFC 7807 `application/problem+json` by the global handlers in `pipelex_api/exception_handlers.py`. **Route handlers do not shape errors themselves** — they call into pipelex and let exceptions propagate. The wire contract is documented at `docs/api-server/error-responses.md`, at the repository root.- **Domain errors** (pipelex `PipelexError` subclasses) — raise from your code and let them propagate. The `PipelexError` global handler obtains an `ErrorReport` via `to_error_report()` and renders it into a problem document. Do not wrap, classify, or re-shape.- **API-authored 4xx/5xx** — use the helpers in `pipelex_api/errors.py`: `raise_validation_error`, `raise_bad_request`, `raise_forbidden`, `raise_unauthenticated`, `raise_payload_too_large`, `raise_internal_server_error`. Each raises an `ApiError` carrying a pre-built problem document; the global handler emits it. **Do not raise `HTTPException` directly** — FastAPI's default handler wraps the body as `{"detail": <whatever>}` and cannot emit a flat RFC 7807 document.- **Auth errors** — the helpers set `WWW-Authenticate: Bearer` automatically on 401.-- **Logging** — the global handlers emit one structured record per error, carrying `event: "api_error"`, `route`, `error_type`, `error_domain`, `retryable`, `status`, and `user_id` / `pipe_code` / `pipeline_run_id` when the request bound them. `detail` rides the API-authored path only: a Pipelex `ErrorReport`'s body text has been through disclosure redaction, so it is not the cause and is deliberately not logged. They travel as `fields=` on the runtime's log call, never interpolated into the message, and `request_id` is not among them: `RequestIdMiddleware` binds it on the runtime's log context, so every record emitted under the request already carries it. The server selects the `json` sink, so a record reaches stderr as one JSON object per line — see `docs/logging.md`. Log disposition follows the final HTTP status: 4xx logs at `warning` (caller mistakes, the provider-429 passthrough, and API-level 4xx overrides like the 409 conflict); 5xx logs at `error` with traceback. Routes should not log error tracebacks themselves.+- **Logging** — the global handlers emit one structured record per error, carrying `event: "api_error"`, `route`, `error_type`, `error_domain`, `retryable`, `status`, and `user_id` / `pipe_code` / `pipeline_run_id` when the request bound them. `detail` rides the API-authored path only: a Pipelex `ErrorReport`'s body text has been through disclosure redaction, so it is not the cause and is deliberately not logged. They travel as `fields=` on the runtime's log call, never interpolated into the message, and `request_id` is not among them: `RequestIdMiddleware` binds it on the runtime's log context, so every record emitted under the request already carries it. The server selects the `json` sink, so a record reaches stderr as one JSON object per line — see `docs/api-server/logging.md` at the repository root. Log disposition follows the final HTTP status: 4xx logs at `warning` (caller mistakes, the provider-429 passthrough, and API-level 4xx overrides like the 409 conflict); 5xx logs at `error` with traceback. Routes should not log error tracebacks themselves.- **Documenting a failure in OpenAPI** — the shared, typed `responses=` declarations live in `pipelex_api/openapi_responses.py` (`ProblemDocument` + one constant per status). Every auth-wrapped `/v1` route already documents `401`/`413`/`422`/`500` via the composite router's `responses=` (`pipelex_api/routes/__init__.py`); a route declares on its own decorator only the statuses **it alone** can produce. Never hand-write an error `content` block: `pipelex_api/openapi_schema.py` re-keys the generated schema onto `application/problem+json`, because FastAPI renders a response `model` under the route's response-class media type and offers no per-response override. Adding a new status means adding a constant there and referencing it — then `make openapi-export`.Typical route:diff --git a/api/Dockerfile b/api/Dockerfileindex d33b6bdab..935201269 100644--- a/api/Dockerfile+++ b/api/Dockerfile@@ -18,13 +18,14 @@ RUN pip install --no-cache-dir uvWORKDIR /app# The lock and the two projects' metadata first, so the layer holding the third-party dependencies-# is cached across source changes. `--package pipelex-api` installs the server's dependency set:-# pipelex with the extras the server's pyproject.toml names, and `cli` deliberately not among them,-# because the server selects the `json` log sink and a Rich-free pretty-print mode, so nothing it does-# on a request renders a terminal. (Rich itself is still installed — typer and instructor both-# require it unconditionally — it is simply never reached.)+# is cached across source changes. The server's metadata includes `hatch_build.py`, the hook that+# writes its dependencies and pins pipelex to the release's version. `--package pipelex-api` installs+# the server's dependency set: pipelex with the extras the server's pyproject.toml names, and `cli`+# deliberately not among them, because the server selects the `json` log sink and a Rich-free+# pretty-print mode, so nothing it does on a request renders a terminal. (Rich itself is still+# installed — typer and instructor both require it unconditionally — it is simply never reached.)COPY pyproject.toml uv.lock README.md LICENSE ./-COPY api/pyproject.toml api/README.md api/LICENSE api/+COPY api/pyproject.toml api/hatch_build.py api/README.md api/LICENSE api/RUN uv sync --frozen --no-dev --package pipelex-api --no-install-workspace# Then the two packages themselves, built as wheels rather than linked to the source tree, so the@@ -54,7 +55,7 @@ ENV PATH="/app/.venv/bin:${PATH}"# ─── Pipelex global config (terms acceptance, backends) ────────────────────────# The server's bundled config becomes ~/.pipelex (the global config dir), and no project-level# .pipelex exists in the image, so users mounting overrides into /root/.pipelex aren't shadowed.-# See api/docs/configuration.md.+# See docs/api-server/configuration.md at the repository root.COPY api/.pipelex/ /root/.pipelex/# ─── runtime config ────────────────────────────────────────────────────────────diff --git a/api/Makefile b/api/Makefileindex b5a40b56c..2ad91e4af 100644--- a/api/Makefile+++ b/api/Makefile@@ -33,17 +33,9 @@ ifneq (,$(wildcard .env))endif# The server takes the library's version, from the root `pyproject.toml` (see this member's `[tool.hatch.version]`).+# Its form is checked where the release is cut, by the root's `version-check.yml` and `prerelease-version-check.yml`.$(eval VERSION=$(shell grep -m1 '^version' ../pyproject.toml | sed -E 's/version = "(.*)"/\1/'))-check-version-format:- @echo "Checking version format: $(VERSION)"- @if ! echo "$(VERSION)" | grep -Eq '^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$$'; then \- echo "Error: Version $(VERSION) is not in the correct format (x.y.z where x, y, z are non-negative integers)."; \- exit 1; \- else \- echo "Version $(VERSION) is valid."; \- fi--include Makefiles/Makefile_basics.mk-include Makefiles/Makefile.local.mkdiff --git a/api/Makefiles/Makefile.deploy.api.mk b/api/Makefiles/Makefile.deploy.api.mkindex 86c610c4b..48fcea274 100644--- a/api/Makefiles/Makefile.deploy.api.mk+++ b/api/Makefiles/Makefile.deploy.api.mk@@ -4,7 +4,8 @@define HELP_DEPLOY_API$(GREEN)Deployment Commands:$(RESET)- make deploy-docker-hub - Deploy the image to Docker Hub.+ make deploy-docker-hub - Build, smoke-test and push the image to Docker Hub (the release workflow runs it).+ make deploy-docker-hub-latest - Point :latest at the already-published :VERSION image, without rebuilding it.endefexport HELP_DEPLOY_API@@ -13,27 +14,64 @@ export HELP_DEPLOY_API############################################################################ Builds the public Pipelex API image and publishes it to Docker Hub as-# pipelex/pipelex-api:$(VERSION) and :latest, with the repository root as the-# build context (see the Dockerfile's header). $(VERSION) is the library's-# version, read from the root pyproject.toml. No workflow runs this target yet:-# publishing the image from pipelex's release is still to be wired, so until-# then a release does not push an image. This repo only publishes to Docker-# Hub — anything beyond (private registries, ECR/ACR/GCR, ECS/k8s deploys) is-# the user's responsibility, typically in a separate infra repo.+# pipelex/pipelex-api:$(VERSION), and as :latest unless PUSH_LATEST=false, with+# the repository root as the build context (see the Dockerfile's header).+# $(VERSION) is the library's version, read from the root pyproject.toml, so the+# image tag names the pipelex it runs. The image is smoke-tested (`docker-smoke`)+# before anything is pushed, so a build that does not boot, or that reports+# another version, never reaches Docker Hub.+#+# The release workflow runs this target from the commit the release's tag names,+# once both distributions are on PyPI: `.github/workflows/publish-docker-hub.yml`,+# which `publish-pypi.yml` calls and which can also be dispatched on its own to+# retry a failed push. It runs this target only when Docker Hub does not serve+# the version yet, since a rebuild would be a different image under the same+# tag, and passes PUSH_LATEST=false unless the version is the newest stable+# release, so `latest` never moves to a pre-release or back to an older+# release. DOCKER_HUB_TOKEN comes from that workflow's secret, through the+# environment: the recipe reads it from the shell rather than expanding it into+# the command line. This repository only publishes to Docker Hub; anything+# beyond (private registries, ECR/ACR/GCR, ECS/k8s deploys) is the user's+# responsibility, typically in a separate infra repo.+PUSH_LATEST ?= true+.PHONY: deploy-docker-hubdeploy-docker-hub:@echo "\n########################### Docker Hub Build ##########################"+ @test -n "$$DOCKER_HUB_TOKEN" || { echo "ERROR: DOCKER_HUB_TOKEN is not set in the environment."; exit 1; }+ @case "$(PUSH_LATEST)" in true|false) ;; *) echo "ERROR: PUSH_LATEST must be true or false, got '$(PUSH_LATEST)'."; exit 1;; esac@echo "Logging in to Docker Hub..."- @echo "$(DOCKER_HUB_TOKEN)" | docker login --username pipelex --password-stdin+ @printf '%s' "$$DOCKER_HUB_TOKEN" | docker login --username pipelex --password-stdin@echo "Building Docker image for Docker Hub..."docker build --platform linux/amd64 -f $(CURDIR)/Dockerfile -t pipelex-api:$(VERSION) $(WORKSPACE_ROOT)- docker tag pipelex-api:$(VERSION) pipelex/pipelex-api:$(VERSION)- docker tag pipelex-api:$(VERSION) pipelex/pipelex-api:latest+ @$(MAKE) --no-print-directory docker-smoke SMOKE_IMAGE=pipelex-api:$(VERSION)@echo "Pushing to Docker Hub..."+ docker tag pipelex-api:$(VERSION) pipelex/pipelex-api:$(VERSION)docker push pipelex/pipelex-api:$(VERSION)- docker push pipelex/pipelex-api:latest-- @echo "✓ Built and pushed image pipelex-api:$(VERSION) and latest to Docker Hub"+ @if [ "$(PUSH_LATEST)" = "true" ]; then \+ docker tag pipelex-api:$(VERSION) pipelex/pipelex-api:latest && \+ docker push pipelex/pipelex-api:latest && \+ echo "✓ Built and pushed pipelex/pipelex-api:$(VERSION) and :latest to Docker Hub"; \+ else \+ echo "✓ Built and pushed pipelex/pipelex-api:$(VERSION) to Docker Hub; :latest left where it was"; \+ fi@echo "✓ Image available at: https://hub.docker.com/r/pipelex/pipelex-api"++# Points pipelex/pipelex-api:latest at the image Docker Hub already serves as+# pipelex/pipelex-api:$(VERSION), copying the manifest registry-side without+# building or pulling anything, so `latest` names exactly the published image.+# `--prefer-index=false` makes it a carbon copy: by default buildx wraps a+# single image manifest, which is what `docker push` publishes here, in a new+# index whose digest differs, and the release workflow's confirmation, which+# requires both tags to name one digest, would then never pass. The release+# workflow runs it on a retry that finds the version tag already pushed, the+# case `deploy-docker-hub` must not rebuild.+.PHONY: deploy-docker-hub-latest+deploy-docker-hub-latest:+ @test -n "$$DOCKER_HUB_TOKEN" || { echo "ERROR: DOCKER_HUB_TOKEN is not set in the environment."; exit 1; }+ @echo "Logging in to Docker Hub..."+ @printf '%s' "$$DOCKER_HUB_TOKEN" | docker login --username pipelex --password-stdin+ docker buildx imagetools create --prefer-index=false --tag pipelex/pipelex-api:latest pipelex/pipelex-api:$(VERSION)+ @echo "✓ pipelex/pipelex-api:latest now names the published pipelex/pipelex-api:$(VERSION)"diff --git a/api/Makefiles/Makefile.local.mk b/api/Makefiles/Makefile.local.mkindex b8e581e1d..9ce26894d 100644--- a/api/Makefiles/Makefile.local.mk+++ b/api/Makefiles/Makefile.local.mk@@ -18,6 +18,7 @@ define HELP_LOCAL$(YELLOW)Docker (closest to what's deployed):$(RESET)make docker-build$(RESET): Build the API image from local source, with the repository root as the build context.make docker-run$(RESET): Build + run the image on http://localhost:8081 (foreground, Ctrl+C to stop).+ make docker-smoke$(RESET): Boot the local image and check /health and the versions /v1/version reports (SMOKE_IMAGE=<tag> for another).make docker-run-hub$(RESET): Pull + run the PUBLISHED Docker Hub image (no local build). Tag: make docker-run-hub HUB_TAG=0.5.0make docker-stop$(RESET): Force-stop the Docker container.make docker-logs$(RESET): Tail Docker container logs.@@ -43,7 +44,7 @@ define HELP_LOCALendefexport HELP_LOCAL-.PHONY: docker-build docker-run docker-run-hub docker-stop docker-logs+.PHONY: docker-build docker-run docker-smoke docker-run-hub docker-stop docker-logs.PHONY: temporal-server ts temporal-server-bare ts-bare tsb temporal-stop tstop.PHONY: check-bundle-arg bundle-run bundle-validate bundle-resolve bundle-codegen bundle-curl bundle-postman bundle-dry@@ -83,6 +84,40 @@ docker-run-hub:--env-file $(ENV_FILE) \$(HUB_IMAGE)+# Boot an image and check that it serves: `/health` answers, and `/v1/version` reports this+# repository's version for both the server and the pipelex it runs, which is the lockstep the release+# promises. Pull-request CI runs it on the image `docker-build` made, and `deploy-docker-hub` runs it+# on the image it is about to push. The gateway key is a placeholder: boot requires one, and neither+# route reaches a provider. SMOKE_IMAGE=<tag> names another image.+SMOKE_IMAGE ?= $(LOCAL_IMAGE)+SMOKE_CONTAINER ?= pipelex-api-smoke+SMOKE_PORT ?= 18081+SMOKE_TIMEOUT ?= 90++docker-smoke:+ @echo "\n=== Smoke-test $(SMOKE_IMAGE): /health, then /v1/version against $(VERSION) ==="+ @docker rm -f $(SMOKE_CONTAINER) >/dev/null 2>&1 || true+ @docker run --detach --platform linux/amd64 --name $(SMOKE_CONTAINER) \+ -p 127.0.0.1:$(SMOKE_PORT):8081 \+ -e PIPELEX_GATEWAY_API_KEY=smoke-test-placeholder \+ $(SMOKE_IMAGE) >/dev/null+ @healthy=""; \+ for attempt in $$(seq 1 $(SMOKE_TIMEOUT)); do \+ if curl -fsS http://127.0.0.1:$(SMOKE_PORT)/health >/dev/null 2>&1; then healthy=1; break; fi; \+ if [ "$$(docker inspect --format '{{.State.Running}}' $(SMOKE_CONTAINER) 2>/dev/null)" != "true" ]; then break; fi; \+ sleep 1; \+ done; \+ if [ -z "$$healthy" ]; then \+ echo "ERROR: $(SMOKE_IMAGE) did not answer /health within $(SMOKE_TIMEOUT)s. Its logs:"; \+ docker logs $(SMOKE_CONTAINER) 2>&1 | tail -60; \+ docker rm -f $(SMOKE_CONTAINER) >/dev/null 2>&1; \+ exit 1; \+ fi; \+ python3 $(CURDIR)/scripts/check_served_version.py http://127.0.0.1:$(SMOKE_PORT) --version $(VERSION); \+ exit_code=$$?; \+ docker rm -f $(SMOKE_CONTAINER) >/dev/null 2>&1; \+ exit $$exit_code+docker-stop:@docker rm -f $(CONTAINER_NAME) 2>/dev/null && echo "Stopped $(CONTAINER_NAME)" || echo "$(CONTAINER_NAME) was not running"diff --git a/api/Makefiles/Makefile_basics.mk b/api/Makefiles/Makefile_basics.mkindex e25f701fc..734dfb545 100644--- a/api/Makefiles/Makefile_basics.mk+++ b/api/Makefiles/Makefile_basics.mk@@ -16,13 +16,17 @@ WORKSPACE_ROOT := $(abspath $(CURDIR)/..)# The "?" is used to make the variable optional, so that it can be overridden by the user.PYTHON_VERSION ?= 3.13+# The Python version pyright and mypy check against, when it is not the environment's own: the root's+# pre-main gate (`lint-fresh-check.yml`) checks every supported version from one 3.11 environment.+LINT_PYTHON_VERSION ?=+# Where `make build` writes the pipelex-api wheel and sdist, and where `make check-lockstep-pin` reads them.+DIST_DIR ?= distVENV_PYTHON := $(VIRTUAL_ENV)/bin/pythonVENV_PYTEST := $(VIRTUAL_ENV)/bin/pytestVENV_RUFF := $(VIRTUAL_ENV)/bin/ruffVENV_PYRIGHT := $(VIRTUAL_ENV)/bin/pyrightVENV_MYPY := $(VIRTUAL_ENV)/bin/mypyVENV_PIPELEX := $(VIRTUAL_ENV)/bin/pipelex-VENV_MKDOCS := $(VIRTUAL_ENV)/bin/mkdocsVENV_PYLINT := $(VIRTUAL_ENV)/bin/pylintUV_MIN_VERSION = $(shell grep -m1 'required-version' $(WORKSPACE_ROOT)/pyproject.toml | sed -E 's/.*= *"([^<>=, ]+).*/\1/')@@ -52,7 +56,8 @@ Usage:make env - Create python virtual envmake lock - Refresh the workspace uv.lock (at the repository root) without updating anythingmake install - Create local virtualenv & install all dependencies-make build - Build the pipelex-api wheel and sdist+make build - Build the pipelex-api wheel and sdist into $(DIST_DIR)+make check-lockstep-pin - Check that the built distributions pin pipelex to this version (after make build)make format - format with ruff formatmake lint - lint with ruff check@@ -90,10 +95,7 @@ make fix-unused-imports - Fix unused imports with ruffmake fui - Shorthand -> fix-unused-importsmake check-TODOs - Check for TODOs-make docs - Serve documentation locally with mkdocs-make docs-check - Check documentation build with mkdocs--make openapi-export - Export the FastAPI OpenAPI schema to docs/openapi/pipelex-api.openapi.yaml+make openapi-export - Export the FastAPI OpenAPI schema to the docs site's docs/api-server/openapi/pipelex-api.openapi.yamlmake openapi-check - Fail if the committed OpenAPI artifact drifts from the appmake kit-sync - Re-sync the vendored .pipelex/inference/ tree from the pipelex kit of this repository@@ -111,7 +113,7 @@ endefexport HELP.PHONY: \- all help env lock install build \+ all help env lock install build check-lockstep-pin \format lint pyright mypy pylint \cleanderived cleanenv cleanall \agent-check agent-test \@@ -120,7 +122,7 @@ export HELPrun-all-tests run-manual-trigger-gha-tests run-gha_disabled-tests \check c cc \merge-check-ruff-lint merge-check-ruff-format merge-check-mypy merge-check-pyright \- li check-unused-imports fix-unused-imports check-uv check-TODOs docs docs-check \+ li check-unused-imports fix-unused-imports check-uv check-TODOs \openapi-export openapi-check \kit-sync kit-check \test-count check-test-badge@@ -169,9 +171,16 @@ lock: env@uv lock && \echo uv lock without update;+# The metadata hook (`hatch_build.py`) pins pipelex to this version in what it builds; the check below is+# what the release workflow runs on its own build before uploading it.build: env$(call PRINT_TITLE,"Building the pipelex-api wheel and sdist")- @uv build --package pipelex-api+ @rm -f $(DIST_DIR)/pipelex_api-*+ @uv build --package pipelex-api --out-dir $(DIST_DIR)++check-lockstep-pin:+ $(call PRINT_TITLE,"Checking the lockstep pin in the distributions in $(DIST_DIR)")+ @python3 scripts/check_lockstep_pin.py $(DIST_DIR)########################################################################################################################## Cleaning ############################@@ -363,11 +372,11 @@ merge-check-ruff-lint: env check-unused-importsmerge-check-pyright: env$(call PRINT_TITLE,"Typechecking with pyright")- $(VENV_PYRIGHT) --pythonpath $(VIRTUAL_ENV)/bin/python3+ $(VENV_PYRIGHT) --pythonpath $(VIRTUAL_ENV)/bin/python3 $(if $(LINT_PYTHON_VERSION),--pythonversion $(LINT_PYTHON_VERSION))merge-check-mypy: env$(call PRINT_TITLE,"Typechecking with mypy")- $(VENV_MYPY) --config-file pyproject.toml+ $(VENV_MYPY) --config-file pyproject.toml $(if $(LINT_PYTHON_VERSION),--python-version $(LINT_PYTHON_VERSION))merge-check-pylint: env$(call PRINT_TITLE,"Linting with pylint")@@ -434,24 +443,13 @@ check: cc check-unused-imports pylint openapi-check kit-checkli: lock install@echo "> done: lock install"-##########################################################################################-### DOCUMENTATION-##########################################################################################--docs: env- $(call PRINT_TITLE,"Serving documentation with mkdocs")- $(VENV_MKDOCS) serve -a 127.0.0.1:8001 --watch docs--docs-check: env- $(call PRINT_TITLE,"Checking documentation build with mkdocs")- $(VENV_MKDOCS) build --strict--############################################################################################# OPENAPI ARTIFACT##########################################################################################-OPENAPI_ARTIFACT := docs/openapi/pipelex-api.openapi.yaml+# The server's pages are part of the pipelex docs site (`docs/api-server/` at the repository root, served by+# the root `make docs`), and the artifact sits among them, so the site publishes it beside the pages that link it.+OPENAPI_ARTIFACT := $(WORKSPACE_ROOT)/docs/api-server/openapi/pipelex-api.openapi.yaml# These depend on `install` (not `env`) on purpose: `env` only ensures the venv# directory exists, it never runs `uv sync`. The schema is generated from whateverdiff --git a/api/README.md b/api/README.mdindex 6660ca3d5..10fce63d5 100644--- a/api/README.md+++ b/api/README.md@@ -6,14 +6,14 @@The official REST API server for building and executing Pipelex pipelines. Deploy your pipelines as HTTP endpoints and integrate them into any application or workflow.<div>- <a href="https://docs.pipelex.com/pages/api/"><strong>API Documentation</strong></a> -+ <a href="https://docs.pipelex.com/latest/api-server/"><strong>API Documentation</strong></a> -<a href="https://github.com/Pipelex/pipelex"><strong>Pipelex Core</strong></a> -<a href="https://go.pipelex.com/discord"><strong>Discord</strong></a></div><br/><p align="center">- <a href="LICENSE"><img src="https://img.shields.io/badge/License-Elastic--2.0-blue.svg" alt="Elastic License 2.0"></a>+ <a href="https://docs.pipelex.com/latest/license/"><img src="https://img.shields.io/badge/License-Elastic--2.0-blue.svg" alt="Elastic License 2.0"></a><a href="https://go.pipelex.com/discord"><img src="https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white" alt="Discord"></a><a href="https://docs.pipelex.com/"><img src="https://img.shields.io/badge/Docs-03bb95?logo=read-the-docs&logoColor=white&style=flat" alt="Documentation"></a></p>@@ -21,8 +21,7 @@ The official REST API server for building and executing Pipelex pipelines. Deplo----> [!IMPORTANT]-> **This server lives in [`Pipelex/pipelex`](https://github.com/Pipelex/pipelex), as the repository's `api/` directory.** It moved there from `Pipelex/pipelex-api` after v0.33.2 and is released together with the pipelex library, under the library's version number, so the tag of the `pipelex/pipelex-api` image that follows 0.33.2 is a pipelex version. The image keeps its name, its port and its `/root/.pipelex` configuration mount. Please open issues on `Pipelex/pipelex`.+> **Released with pipelex, under pipelex's version.** This server is the [`api/` directory](https://github.com/Pipelex/pipelex/tree/main/api) of [`Pipelex/pipelex`](https://github.com/Pipelex/pipelex), and every pipelex release ships it: the `pipelex-api` package on PyPI pins the `pipelex` of the same version, and `pipelex/pipelex-api:X.Y.Z` runs pipelex X.Y.Z. It was released on its own from `Pipelex/pipelex-api` until v0.33.2, so the image tag that follows 0.33.2 is a pipelex version. The image keeps its name, its port and its `/root/.pipelex` configuration mount. Please open issues on `Pipelex/pipelex`.# 📑 Table of Contents@@ -38,13 +37,13 @@ The official REST API server for building and executing Pipelex pipelines. DeploThe **Pipelex API Server** is a FastAPI-based REST API that allows you to execute [Pipelex](https://github.com/Pipelex/pipelex) pipelines via HTTP requests. Deploy your pipelines as HTTP endpoints and integrate them into any application or workflow.-It is the source-available reference implementation of the **[MTHDS Protocol](https://mthds.ai)** — the minimal HTTP contract every MTHDS runner implements (`POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version`). The contracts nest: **MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API**. This server adds the build tooling extensions (`/build/*`) on top of the protocol; the hosted API at `api.pipelex.com/v1` adds durable runs, the method catalog, and account management on top of this server — same shapes throughout. All routes live under the `/v1` base path; the committed contract is [`docs/openapi/pipelex-api.openapi.yaml`](docs/openapi/pipelex-api.openapi.yaml).+It is the source-available reference implementation of the **[MTHDS Protocol](https://mthds.ai)** — the minimal HTTP contract every MTHDS runner implements (`POST /execute`, `POST /start`, `POST /validate`, `GET /models`, `GET /version`). The contracts nest: **MTHDS Protocol ⊂ Pipelex API (this server) ⊂ Pipelex hosted API**. This server adds the build tooling extensions (`/build/*`) on top of the protocol; the hosted API at `api.pipelex.com/v1` adds durable runs, the method catalog, and account management on top of this server — same shapes throughout. All routes live under the `/v1` base path; the committed contract is [`pipelex-api.openapi.yaml`](https://github.com/Pipelex/pipelex/blob/main/docs/api-server/openapi/pipelex-api.openapi.yaml).# 🚀 Quick Start with Docker**Official Docker image available at:** [`pipelex/pipelex-api`](https://hub.docker.com/r/pipelex/pipelex-api)-The published image is **generic and configuration-light**: Temporal is off, no S3, no remote tracing. It boots with a single required env var (`PIPELEX_GATEWAY_API_KEY`), and you bring your own [Pipelex configuration](docs/configuration.md) on top to enable storage, tracing, Temporal, or anything else.+The published image is **generic and configuration-light**: Temporal is off, no S3, no remote tracing. It boots with a single required env var (`PIPELEX_GATEWAY_API_KEY`), and you bring your own [Pipelex configuration](https://docs.pipelex.com/latest/api-server/configuration/) on top to enable storage, tracing, Temporal, or anything else.### 1. Run with Docker@@ -56,7 +55,7 @@ docker run --name pipelex-api -p 8081:8081 \pipelex/pipelex-api:latest```-To require authentication on the API, add `-e AUTH_MODE=api_key -e API_KEY=your-secret` (or `AUTH_MODE=jwt` + `JWT_SECRET_KEY`). See [`.env.example`](.env.example) for the full list of supported variables and [docs/configuration.md](docs/configuration.md) for `--env-file` and `docker compose` patterns if you'd rather keep config out of your shell history.+To require authentication on the API, add `-e AUTH_MODE=api_key -e API_KEY=your-secret` (or `AUTH_MODE=jwt` + `JWT_SECRET_KEY`). See [`.env.example`](https://github.com/Pipelex/pipelex/blob/main/api/.env.example) for the full list of supported variables and the [Configuration page](https://docs.pipelex.com/latest/api-server/configuration/) for `--env-file` and `docker compose` patterns if you'd rather keep config out of your shell history.If you'd rather build the image yourself instead of pulling, replace `pipelex/pipelex-api:latest` with a local tag after `docker build -f api/Dockerfile -t pipelex-api .`, run from the root of a `Pipelex/pipelex` checkout: the build context is the repository root, so the image installs the pipelex library of the same commit.@@ -66,7 +65,7 @@ If you'd rather build the image yourself instead of pulling, replace `pipelex/picurl http://localhost:8081/health```-The API is now running at `http://localhost:8081`. To customize behavior (enable Temporal, swap to S3 storage, layer in env-specific overrides, …), see [docs/configuration.md](docs/configuration.md).+The API is now running at `http://localhost:8081`. To customize behavior (enable Temporal, swap to S3 storage, layer in env-specific overrides, …), see the [Configuration page](https://docs.pipelex.com/latest/api-server/configuration/).# 🧪 Run your first pipeline@@ -98,7 +97,7 @@ You'll get back a JSON response with `state: "COMPLETED"` and the summary under`Document` accepts public HTTP/HTTPS URLs, `pipelex-storage://` URIs, or base64 data URLs. For images, use the `Image` concept with the same `{ "url": "..." }` shape.-For inline MTHDS in the request, `mthds_contents` is a JSON array of raw `.mthds` (TOML) file contents as strings — typically `[open("my_pipe.mthds").read()]` from a client. See [docs/pipe-run.md](docs/pipe-run.md) for every supported input shape and the full `/execute` reference.+For inline MTHDS in the request, `mthds_contents` is a JSON array of raw `.mthds` (TOML) file contents as strings — typically `[open("my_pipe.mthds").read()]` from a client. See the [Pipe Run page](https://docs.pipelex.com/latest/api-server/pipe-run/) for every supported input shape and the full `/execute` reference.# 📈 How to scale Pipelex@@ -109,32 +108,32 @@ For production-scale workloads (high concurrency, long-running pipelines, retrie- Pipeline runs become durable workflows — survive worker crashes, support retries and timeouts out of the box.- The API container becomes a thin orchestrator: it submits workflows to a Temporal cluster and returns a `pipeline_run_id` immediately (this is what `POST /v1/start` already does).- Pipeline execution itself runs on a separate pool of **Pipelex workers** that you scale independently from the HTTP layer.-- Async completion callbacks (`callback_urls` + `X-Completion-Signature`, see [pipe-run.md](docs/pipe-run.md)) let your application be notified when each run finishes, without polling.+- Async completion callbacks (`callback_urls` + `X-Completion-Signature`, see [Pipe Run](https://docs.pipelex.com/latest/api-server/pipe-run/)) let your application be notified when each run finishes, without polling.Pipelex already integrates with Temporal under the hood, and the Docker image accepts `TEMPORAL_API_KEY` plus a `[temporal] is_enabled = true` override in `.pipelex/`. **A complete deployment recipe (Temporal cluster sizing, worker container, autoscaling guidance, and an end-to-end docker-compose) is coming soon.** In the meantime, if you need to scale today, get in touch on [Discord](https://go.pipelex.com/discord) and we'll help you wire it up.# 📖 API Documentation-The full reference for this API server lives next to the code in [`docs/`](docs/):+The full reference for this API server is part of the Pipelex documentation, under [API Server](https://docs.pipelex.com/latest/api-server/):-- [Overview](docs/index.md) — endpoints, authentication, deployment-- [Pipe Run](docs/pipe-run.md) — `/execute`, `/start`, every input shape-- [Pipe Validate](docs/pipe-validate.md) — `/validate`-- [Pipe Builder](docs/pipe-builder.md) — `/build/inputs`, `/build/output`, `/build/runner`-- [Configuration](docs/configuration.md) — env vars, mounting your own `.pipelex/` config+- [Overview](https://docs.pipelex.com/latest/api-server/) — endpoints, authentication, deployment+- [Pipe Run](https://docs.pipelex.com/latest/api-server/pipe-run/) — `/execute`, `/start`, every input shape+- [Pipe Validate](https://docs.pipelex.com/latest/api-server/pipe-validate/) — `/validate`+- [Pipe Builder](https://docs.pipelex.com/latest/api-server/pipe-builder/) — `/build/inputs`, `/build/output`, `/build/runner`+- [Configuration](https://docs.pipelex.com/latest/api-server/configuration/) — env vars, mounting your own `.pipelex/` configFor broader Pipelex documentation (MTHDS language, concepts, pipe types, the Gateway): **[https://docs.pipelex.com/](https://docs.pipelex.com/)**# 💬 Support-- **API Documentation**: [https://docs.pipelex.com/pages/api/](https://docs.pipelex.com/pages/api/)+- **API Documentation**: [https://docs.pipelex.com/latest/api-server/](https://docs.pipelex.com/latest/api-server/)- **Pipelex Documentation**: [https://docs.pipelex.com/](https://docs.pipelex.com/)- **Discord Community**: [https://go.pipelex.com/discord](https://go.pipelex.com/discord)- **Main Repository**: [https://github.com/Pipelex/pipelex](https://github.com/Pipelex/pipelex)# 📝 License-This project is licensed under the Elastic License 2.0 (ELv2); see [LICENSE](LICENSE) for the terms, and the [license page](https://docs.pipelex.com/latest/license/) for how Pipelex reads them. Runtime dependencies are distributed under their own licenses via PyPI.+This project is licensed under the Elastic License 2.0 (ELv2); see [LICENSE](https://github.com/Pipelex/pipelex/blob/main/LICENSE) for the terms, and the [license page](https://docs.pipelex.com/latest/license/) for how Pipelex reads them. Runtime dependencies are distributed under their own licenses via PyPI.---diff --git a/api/docs/CODE_OF_CONDUCT.md b/api/docs/CODE_OF_CONDUCT.mddeleted file mode 100644index 7e3ea63d8..000000000--- a/api/docs/CODE_OF_CONDUCT.md+++ /dev/null@@ -1,5 +0,0 @@-----title: Code of Conduct--------8<-- "CODE_OF_CONDUCT.md"\ No newline at end of filediff --git a/api/docs/changelog.md b/api/docs/changelog.mddeleted file mode 100644index ece2e41b0..000000000--- a/api/docs/changelog.md+++ /dev/null@@ -1,5 +0,0 @@-----title: Changelog--------8<-- "CHANGELOG.md"\ No newline at end of filediff --git a/api/docs/contributing.md b/api/docs/contributing.mddeleted file mode 100644index ec207150d..000000000--- a/api/docs/contributing.md+++ /dev/null@@ -1,5 +0,0 @@-----title: Contributing to Pipelex--------8<-- "CONTRIBUTING.md"diff --git a/api/docs/license.md b/api/docs/license.mddeleted file mode 100644index 3ddc5d7b5..000000000--- a/api/docs/license.md+++ /dev/null@@ -1,11 +0,0 @@-----title: License-hide:-- feedback------# License--```---8<-- "LICENSE"-```\ No newline at end of filediff --git a/api/docs/stylesheets/general.css b/api/docs/stylesheets/general.cssdeleted file mode 100644index 7446c0fa8..000000000--- a/api/docs/stylesheets/general.css+++ /dev/null@@ -1,75 +0,0 @@-/* ===== Pipelex Docs – globals.css (final & minimal) ===== */--/* Brand colors */-:root {- --md-accent-fg-color: #45BF9F;- --md-primary-fg-color: #45BF9F;- --md-primary-fg-color--light: #6CD4B6;- --md-primary-fg-color--dark: #2E9A7F;-}--/* Utilities */-.equal-width { width: 100%; max-width: 300px; display: inline-block; }-pre code { white-space: pre-wrap !important; word-wrap: break-word; overflow-wrap: break-word; }--/* --- Sidebar nav --- */--/* Ensure all link and label elements have no shadow */-.md-nav__link,-label.md-nav__link {- box-shadow: none !important;-}--/* 1) Baseline: non-active links = light grey (declare ONCE) */-.md-nav--primary .md-nav__link:not(.md-nav__link--active) {- color: rgba(0, 0, 0, 0.6) !important;-}-[data-md-color-scheme="slate"] .md-nav--primary .md-nav__link:not(.md-nav__link--active) {- color: rgba(255, 255, 255, 0.4) !important;-}--/* 2) TOP-LEVEL SECTION HEADERS ONLY (dark & bold)- Works across Material versions:- - first-level <li> that contains a child <nav>- - OR first-level <li> that contains the toggle input- - OR items explicitly marked as section by Material */-.md-sidebar__inner nav.md-nav--primary > ul.md-nav__list- > li.md-nav__item:has(> nav) > label.md-nav__link,-.md-sidebar__inner nav.md-nav--primary > ul.md-nav__list- > li.md-nav__item:has(> input.md-nav__toggle) > label.md-nav__link,-.md-nav--primary .md-nav__item.md-nav__item--section > label.md-nav__link {- color: #000 !important;- font-weight: 700 !important;-}-[data-md-color-scheme="slate"] .md-sidebar__inner nav.md-nav--primary > ul.md-nav__list- > li.md-nav__item:has(> nav) > label.md-nav__link,-[data-md-color-scheme="slate"] .md-sidebar__inner nav.md-nav--primary > ul.md-nav__list- > li.md-nav__item:has(> input.md-nav__toggle) > label.md-nav__link,-[data-md-color-scheme="slate"] .md-nav--primary .md-nav__item.md-nav__item--section > label.md-nav__link {- color: #fff !important;- font-weight: 700 !important;-}-/* Constant, small pill padding on ALL items (links + labels) */-.md-nav--primary .md-nav__link,-.md-nav--primary label.md-nav__link {- padding: 6px 10px; /* tighter than before */- margin: 0px 0px;- border-radius: 10px; /* softer, not huge */- box-sizing: border-box;- transition: background-color 120ms ease, color 120ms ease; /* color only, no layout props */-}--/* Hover: color only (no padding/radius changes) */-.md-nav--primary a.md-nav__link:not(.md-nav__link--active):hover {- background-color: rgba(255, 255, 255, 0.08);-}-[data-md-color-scheme="default"] .md-nav--primary a.md-nav__link:not(.md-nav__link--active):hover {- background-color: rgba(0, 0, 0, 0.06);-}--/* Active: color only (no padding/radius changes) */-.md-nav--primary .md-nav__link--active {- color: #45BF9F !important;- font-weight: 700 !important;- background-color: rgba(69, 191, 159, 0.12) !important;-}diff --git a/api/docs/stylesheets/mermaid.css b/api/docs/stylesheets/mermaid.cssdeleted file mode 100644index d9052745b..000000000--- a/api/docs/stylesheets/mermaid.css+++ /dev/null@@ -1,36 +0,0 @@-/* docs/stylesheets/mermaid.css- Tweak the colours Material exposes for Mermaid */--.mermaid {- --md-mermaid-edge-color: #000000;- --md-mermaid-label-bg-color: #ffdbfa;- --md-mermaid-label-fg-color: #000000;-- --md-mermaid-node-bg-color: #fff9db;- --md-mermaid-node-fg-color: #000000;-}--/* Give the diagram itself a code-block like backdrop so it stands out- on both light and dark themes. */-.md-typeset .mermaid {- padding: 0.3rem;- border-radius: 0.3rem;- overflow: auto;-}--/* --- Light color scheme ---------------------------------------------- */--[data-md-color-scheme="default"] .md-typeset .mermaid {- background-color: hsla(140, 27%, 98%, 0.983);-}--/* --- Dark color scheme ------------------------------------------------ */--[data-md-color-scheme="slate"] .md-typeset .mermaid {- background-color: hsla(140, 27%, 98%, 0.983);-}--/* Ensure text inside nodes and labels is black */-.mermaid text {- fill: #000 !important;-}\ No newline at end of filediff --git a/api/hatch_build.py b/api/hatch_build.pynew file mode 100644index 000000000..24eae7250--- /dev/null+++ b/api/hatch_build.py@@ -0,0 +1,48 @@+"""The hatch metadata hook that pins the published server on the pipelex it was released with.++The server and the library are released together under one version number, which this member reads from the+repository root's `pyproject.toml` (`[tool.hatch.version]`). In the workspace, its `pipelex` dependency resolves+from the same commit and carries no version, because uv ignores a version specifier on a workspace source. A+published `pipelex-api` has no workspace to resolve from, so its metadata must say which pipelex it was released+with, or an install would pair the server with whatever pipelex PyPI serves.++This hook writes the dependencies: each requirement of `lockstep-dependencies` pinned to `==` this distribution's+own version, then `dependencies` as written. Both lists live in `[tool.hatch.metadata.hooks.custom]` of+`pyproject.toml`, and `dependencies` is declared dynamic there, since a build backend may not rewrite a static+field. A wheel built from the sdist reads the dependencies back from `PKG-INFO`, so this hook runs once per release.+"""++from __future__ import annotations++from typing import Any++from hatchling.metadata.plugin.interface import MetadataHookInterface+from packaging.requirements import Requirement+++class LockstepPinMetadataHook(MetadataHookInterface):+ def update(self, metadata: dict[str, Any]) -> None:+ version = metadata.get("version")+ if not isinstance(version, str) or not version:+ msg = "The lockstep pin needs the distribution's version, which hatch had not resolved when the hook ran."+ raise ValueError(msg)+ lockstep_requirements = self._string_list(option="lockstep-dependencies")+ other_requirements = self._string_list(option="dependencies")+ pinned_requirements: list[str] = []+ for requirement_text in lockstep_requirements:+ requirement = Requirement(requirement_text)+ if requirement.specifier or requirement.url or requirement.marker:+ msg = (+ f"`{requirement_text}` in `lockstep-dependencies` must name the package and its extras only: "+ "the hook supplies the version, so a specifier, a URL or a marker written here would be overridden."+ )+ raise ValueError(msg)+ pinned_requirements.append(f"{requirement_text}=={version}")+ metadata["dependencies"] = [*pinned_requirements, *other_requirements]++ def _string_list(self, *, option: str) -> list[str]:+ value = self.config.get(option)+ if not isinstance(value, list) or not all(isinstance(item, str) for item in value):+ msg = f"Option `{option}` of the custom metadata hook must be a list of strings."+ raise TypeError(msg)+ return list(value)diff --git a/api/mkdocs.yml b/api/mkdocs.ymldeleted file mode 100644index eed60b07b..000000000--- a/api/mkdocs.yml+++ /dev/null@@ -1,124 +0,0 @@-site_name: Pipelex API Documentation--site_url: https://pipelex.github.io/pipelex-api/-site_description: "Official API documentation for Pipelex - REST API for executing Pipelex pipelines and validating MTHDS files"-docs_dir: docs-repo_url: "https://github.com/Pipelex/pipelex"-repo_name: "Pipelex on GitHub"--# `CONTRIBUTING.md` lives at the repository root, one level above this member, and is snippet-included into-# `docs/contributing.md`. Its links (e.g. `docs/configuration.md`) are-# authored to work from the repo root on GitHub; from mkdocs' perspective-# the included content sits inside `docs/` and the same relative paths-# can't resolve. Downgrade these specific link checks from `warn`-# (which `mkdocs build --strict` treats as fatal) to `info` so we keep-# the human-friendly GitHub links without the build failing.-validation:- links:- not_found: info- unrecognized_links: info--theme:- name: material- favicon: https://pipelex-web.s3.amazonaws.com/logo/pipelex-favicon-32.png- logo: https://pipelex-web.s3.amazonaws.com/logo/Pipelex-logo-wot-mono-560x176.png- features:- - navigation.footer- - navigation.tabs- - navigation.tabs.sticky- - navigation.sections- - navigation.top- palette:- - scheme: default- primary: teal- accent: deep purple- toggle:- icon: material/weather-night- name: Switch to dark mode- - scheme: slate- primary: teal- accent: purple- toggle:- icon: material/weather-sunny- name: Switch to light mode--copyright: "© 2025-2026 Evotis S.A.S. <br/>Pipelex is a trademark of Evotis S.A.S."--extra:- social:- - icon: fontawesome/brands/github- link: https://github.com/Pipelex/pipelex- name: Pipelex on GitHub- - icon: fontawesome/brands/python- link: https://pypi.org/project/pipelex/- name: pipelex on PyPI- - icon: fontawesome/brands/twitter- link: https://x.com/PipelexAI- name: Pipelex on X- - icon: fontawesome/brands/discord- link: https://go.pipelex.com/discord- name: Pipelex on Discord- generator: false--plugins:- - search- - glightbox:- touchNavigation: true- loop: false- effect: zoom- slide_effect: slide- width: 100%- height: auto- zoomable: true- draggable: true- auto_caption: false- caption_position: bottom--markdown_extensions:- - meta- - attr_list- - md_in_html- - pymdownx.blocks.caption- - pymdownx.highlight- - pymdownx.superfences:- custom_fences:- - name: mermaid- class: mermaid- format: !!python/name:pymdownx.superfences.fence_code_format- - pymdownx.emoji:- emoji_index: !!python/name:material.extensions.emoji.twemoji- emoji_generator: !!python/name:material.extensions.emoji.to_svg- # Snippets resolve in this member first (its CHANGELOG.md and LICENSE), then at the repository root,- # which holds the CONTRIBUTING.md and CODE_OF_CONDUCT.md that now cover the server too.- - pymdownx.snippets:- base_path: [".", ".."]- check_paths: true- - admonition- - pymdownx.details--extra_css:- - stylesheets/mermaid.css- - stylesheets/general.css--nav:- - Home: https://docs.pipelex.com/- - API:- - Overview: index.md- - Configuration: configuration.md- - Pipe Builder: pipe-builder.md- - Pipe Run: pipe-run.md- - Pipe Validate: pipe-validate.md- - Pipe I/O: pipe-io.md- - Resolve & Codegen: codegen.md- - MTHDS Tools: mthds-tools.md- - Storage Transport: storage-transport.md- - Error Responses: error-responses.md- - Logging: logging.md- - Contribute:- - Contributing: contributing.md- - Code of Conduct: CODE_OF_CONDUCT.md- - License: license.md- - Changelog: changelog.md- - n8n: https://pipelex.github.io/n8n-nodes-pipelex/- - Contribute: https://docs.pipelex.com/pages/contributing/- - Changelog: https://docs.pipelex.com/pages/changelog/diff --git a/api/pyproject.toml b/api/pyproject.tomlindex b3c49af64..54d3b3318 100644--- a/api/pyproject.toml+++ b/api/pyproject.toml@@ -1,8 +1,9 @@[project]name = "pipelex-api"# The server is released with the library under one version number, read from the root `pyproject.toml`-# (`[tool.hatch.version]` below), so a release bumps one file.-dynamic = ["version"]+# (`[tool.hatch.version]` below), so a release bumps one file. The dependencies are dynamic too: they are+# written in `[tool.hatch.metadata.hooks.custom]` below, whose hook pins pipelex to that version.+dynamic = ["version", "dependencies"]description = "Pipelex API"authors = [{ name = "Evotis S.A.S.", email = "[email protected]" }]maintainers = [{ name = "Pipelex staff", email = "[email protected]" }]@@ -19,19 +20,6 @@ classifiers = ["Programming Language :: Python :: 3.14",]-dependencies = [- # The extras deliberately leave out `cli`: the runner selects the `json` log sink and a Rich-free- # pretty-print mode, so nothing it does on a request renders a terminal.- #- # pipelex resolves from this repository's workspace (`[tool.uv.sources]` below), so the server always- # runs the library of the same commit, and no version is written here: uv ignores a version specifier- # on a workspace source, so a pin would only go stale.- "pipelex[mistralai,anthropic,google,google-genai,bedrock,fal]",- "fastapi>=0.118.0",- "pyjwt>=2.10.1",- "uvicorn>=0.37.0",-]-[build-system]requires = ["hatchling>=1.27"]build-backend = "hatchling.build"@@ -39,8 +27,8 @@ build-backend = "hatchling.build"[project.urls]Homepage = "https://pipelex.com"Repository = "https://github.com/Pipelex/pipelex"-Documentation = "https://docs.pipelex.com/pages/api/"-Changelog = "https://docs.pipelex.com/changelog/"+Documentation = "https://docs.pipelex.com/latest/api-server/"+Changelog = "https://docs.pipelex.com/latest/changelog/"[project.optional-dependencies]dev = [@@ -71,14 +59,6 @@ dev = ["types-PyYAML>=6.0.12.20250326","types-toml>=0.10.8.20240310",]-# One lock resolves every workspace member's extras together, so these floors follow pipelex's own `docs`-# extra rather than pinning versions of their own.-docs = [- "mkdocs>=1.6.1",- "mkdocs-glightbox>=0.4.0",- "mkdocs-material>=9.7.7",- "mkdocs-meta-manager>=1.1.0",-][tool.hatch.metadata]allow-direct-references = true@@ -93,10 +73,35 @@ packages = ["pipelex_api"]path = "../pyproject.toml"pattern = '(?m)^version = "(?P<version>[^"]+)"'+# The server's dependencies, which `hatch_build.py` writes into the distribution's metadata. Each requirement+# of `lockstep-dependencies` is pinned to `==` the release's own version, so a published `pipelex-api` installs+# exactly the pipelex it was released with; `dependencies` is copied as written. In the workspace, pipelex+# resolves from the same commit (`[tool.uv.sources]` below) and uv ignores the pin, so nothing here goes stale+# between releases. The release workflow checks the pin in the built wheel before it uploads it.+[tool.hatch.metadata.hooks.custom]+path = "hatch_build.py"+# The extras deliberately leave out `cli`: the runner selects the `json` log sink and a Rich-free pretty-print+# mode, so nothing it does on a request renders a terminal.+lockstep-dependencies = [+ "pipelex[mistralai,anthropic,google,google-genai,bedrock,fal]",+]+dependencies = [+ # FastAPI 0.142 changed how included routers are mounted, and the server's router assembly+ # (`pipelex_api/main.py`) loses every `/v1` route under it. The ceiling stays until that assembly is+ # adapted: the lock alone cannot hold it, since a published wheel installs whatever its metadata allows.+ "fastapi>=0.118.0,<0.142",+ "pyjwt>=2.10.1",+ "uvicorn>=0.37.0",+]+[tool.uv]-# The version comes from the root `pyproject.toml`, so uv has to rebuild this member's metadata when that-# file changes, not only when this one does.-cache-keys = [{ file = "pyproject.toml" }, { file = "../pyproject.toml" }]+# The version comes from the root `pyproject.toml` and the dependencies from `hatch_build.py`, so uv has to+# rebuild this member's metadata when either changes, not only when this file does.+cache-keys = [+ { file = "pyproject.toml" },+ { file = "../pyproject.toml" },+ { file = "hatch_build.py" },+][tool.uv.sources]pipelex = { workspace = true }diff --git a/api/scripts/check_lockstep_pin.py b/api/scripts/check_lockstep_pin.pynew file mode 100644index 000000000..a01dff6c5--- /dev/null+++ b/api/scripts/check_lockstep_pin.py@@ -0,0 +1,188 @@+"""Check that built `pipelex-api` distributions pin pipelex to the release's own version.++The server is released with the library under one version number, and a published `pipelex-api` must install+exactly the pipelex it was released with. `hatch_build.py` writes that pin into the metadata; this script is the+check the release workflow runs on the built wheel and sdist before uploading them, and that pull-request CI runs+on a test build, so a pin that went missing or wrong never reaches PyPI.++For each requirement of `lockstep-dependencies` (in `[tool.hatch.metadata.hooks.custom]` of this member's+`pyproject.toml`), the wheel's `METADATA` and the sdist's `PKG-INFO` must each carry exactly one unconditional+`Requires-Dist` for that package, with the same extras and the specifier `==X.Y.Z`. Both must also declare+`Version: X.Y.Z`. The version defaults to the one in the repository root's `pyproject.toml`.++It needs nothing but the standard library, so it runs on a bare runner without the member's environment.++Usage:+ python api/scripts/check_lockstep_pin.py dist/+ python api/scripts/check_lockstep_pin.py dist/ --version 0.71.0+"""++import argparse+import re+import sys+import tarfile+import tomllib+import zipfile+from email.parser import Parser+from pathlib import Path+from typing import NamedTuple++_MEMBER_ROOT = Path(__file__).resolve().parents[1]+_REPOSITORY_ROOT = _MEMBER_ROOT.parent+_DISTRIBUTION_FILE_PREFIX = "pipelex_api-"++_REQUIREMENT_PATTERN = re.compile(+ r"""^\s*+ (?P<name>[A-Za-z0-9][A-Za-z0-9._-]*)\s*+ (?:\[(?P<extras>[^\]]*)\])?\s*+ (?P<specifier>[^;]*?)\s*+ (?:;\s*(?P<marker>.*?))?\s*$""",+ re.VERBOSE,+)+++class ParsedRequirement(NamedTuple):+ name: str+ extras: frozenset[str]+ specifier: str+ marker: str+++def normalize_name(name: str) -> str:+ return re.sub(r"[-_.]+", "-", name).lower()+++def parse_requirement(requirement_text: str) -> ParsedRequirement:+ match = _REQUIREMENT_PATTERN.match(requirement_text)+ if match is None:+ msg = f"Cannot parse the requirement `{requirement_text}`."+ raise ValueError(msg)+ extras_text = match.group("extras") or ""+ extras = frozenset(normalize_name(extra.strip()) for extra in extras_text.split(",") if extra.strip())+ return ParsedRequirement(+ name=normalize_name(match.group("name")),+ extras=extras,+ specifier=re.sub(r"\s+", "", match.group("specifier") or "").removeprefix("(").removesuffix(")"),+ marker=(match.group("marker") or "").strip(),+ )+++def read_root_version() -> str:+ with (_REPOSITORY_ROOT / "pyproject.toml").open("rb") as root_pyproject:+ version = tomllib.load(root_pyproject)["project"]["version"]+ if not isinstance(version, str):+ msg = "The repository root's `pyproject.toml` declares no string `[project] version`."+ raise TypeError(msg)+ return version+++def read_lockstep_requirements() -> list[ParsedRequirement]:+ with (_MEMBER_ROOT / "pyproject.toml").open("rb") as member_pyproject:+ hook_config = tomllib.load(member_pyproject)["tool"]["hatch"]["metadata"]["hooks"]["custom"]+ requirement_texts = hook_config["lockstep-dependencies"]+ if not isinstance(requirement_texts, list) or not requirement_texts:+ msg = "`lockstep-dependencies` in the member's `pyproject.toml` must be a non-empty list."+ raise TypeError(msg)+ return [parse_requirement(str(requirement_text)) for requirement_text in requirement_texts]+++def check_metadata(+ *,+ metadata_text: str,+ source: str,+ version: str,+ lockstep_requirements: list[ParsedRequirement],+) -> list[str]:+ """Return what is wrong with one distribution's core metadata, or nothing when it pins correctly."""+ metadata = Parser().parsestr(metadata_text)+ problems: list[str] = []+ declared_version = metadata.get("Version")+ if declared_version != version:+ problems.append(f"{source} declares `Version: {declared_version}`, expected `{version}`.")+ requires_dist = [parse_requirement(str(line)) for line in metadata.get_all("Requires-Dist") or []]+ for expected in lockstep_requirements:+ unconditional = [requirement for requirement in requires_dist if requirement.name == expected.name and not requirement.marker]+ if len(unconditional) != 1:+ problems.append(f"{source} carries {len(unconditional)} unconditional `Requires-Dist` for `{expected.name}`, expected exactly one.")+ continue+ found = unconditional[0]+ if found.specifier != f"=={version}":+ problems.append(f"{source} requires `{expected.name}` with `{found.specifier or 'no version'}`, expected `=={version}`.")+ if found.extras != expected.extras:+ problems.append(f"{source} requires `{expected.name}` with the extras {sorted(found.extras)}, expected {sorted(expected.extras)}.")+ return problems+++def read_wheel_metadata(wheel_path: Path) -> str:+ with zipfile.ZipFile(wheel_path) as wheel:+ metadata_names = [name for name in wheel.namelist() if name.endswith(".dist-info/METADATA") and name.count("/") == 1]+ if len(metadata_names) != 1:+ msg = f"{wheel_path.name} holds {len(metadata_names)} top-level `.dist-info/METADATA` files, expected one."+ raise ValueError(msg)+ return wheel.read(metadata_names[0]).decode("utf-8")+++def read_sdist_metadata(sdist_path: Path) -> str:+ with tarfile.open(sdist_path, "r:gz") as sdist:+ pkg_info_members = [member for member in sdist.getmembers() if member.name.endswith("/PKG-INFO") and member.name.count("/") == 1]+ if len(pkg_info_members) != 1:+ msg = f"{sdist_path.name} holds {len(pkg_info_members)} top-level `PKG-INFO` files, expected one."+ raise ValueError(msg)+ pkg_info = sdist.extractfile(pkg_info_members[0])+ if pkg_info is None:+ msg = f"{sdist_path.name}'s `PKG-INFO` is not a regular file."+ raise ValueError(msg)+ return pkg_info.read().decode("utf-8")+++def check_distributions(*, dist_dir: Path, version: str, lockstep_requirements: list[ParsedRequirement]) -> list[str]:+ """Return what is wrong with the `pipelex-api` wheel and sdist in `dist_dir`, or nothing when both pin correctly."""+ wheels = sorted(dist_dir.glob(f"{_DISTRIBUTION_FILE_PREFIX}*.whl"))+ sdists = sorted(dist_dir.glob(f"{_DISTRIBUTION_FILE_PREFIX}*.tar.gz"))+ if len(wheels) != 1 or len(sdists) != 1:+ found = ", ".join(path.name for path in [*wheels, *sdists]) or "nothing"+ return [f"Expected exactly one `pipelex-api` wheel and one sdist in {dist_dir}, found: {found}."]+ problems: list[str] = []+ for distribution, read_metadata, expected_name in (+ (wheels[0], read_wheel_metadata, f"{_DISTRIBUTION_FILE_PREFIX}{version}-py3-none-any.whl"),+ (sdists[0], read_sdist_metadata, f"{_DISTRIBUTION_FILE_PREFIX}{version}.tar.gz"),+ ):+ if distribution.name != expected_name:+ problems.append(f"{distribution.name} is not a {version} distribution: expected {expected_name}.")+ try:+ metadata_text = read_metadata(distribution)+ except (OSError, ValueError, zipfile.BadZipFile, tarfile.TarError) as exc:+ problems.append(f"Cannot read the metadata of {distribution.name}: {exc}")+ continue+ problems.extend(+ check_metadata(+ metadata_text=metadata_text,+ source=distribution.name,+ version=version,+ lockstep_requirements=lockstep_requirements,+ )+ )+ return problems+++def main() -> int:+ parser = argparse.ArgumentParser(description="Check that built pipelex-api distributions pin pipelex to the release's version.")+ parser.add_argument("dist_dir", type=Path, help="The directory holding the built pipelex-api wheel and sdist")+ parser.add_argument("--version", help="The release's version (default: the repository root's pyproject.toml)")+ args = parser.parse_args()++ version: str = args.version or read_root_version()+ lockstep_requirements = read_lockstep_requirements()+ problems = check_distributions(dist_dir=args.dist_dir, version=version, lockstep_requirements=lockstep_requirements)+ if problems:+ print("Lockstep pin check FAILED:")+ for problem in problems:+ print(f" - {problem}")+ return 1+ pins = ", ".join(f"{requirement.name}=={version}" for requirement in lockstep_requirements)+ print(f"The pipelex-api wheel and sdist in {args.dist_dir} are {version} and pin {pins}.")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/api/scripts/check_served_version.py b/api/scripts/check_served_version.pynew file mode 100644index 000000000..46d4aaad9--- /dev/null+++ b/api/scripts/check_served_version.py@@ -0,0 +1,46 @@+"""Check that a running server reports the release's version for itself and for the pipelex it runs.++The server and the library are released together under one version number, and the image is built from the+release commit, so `GET /v1/version` on a freshly built image must report that number twice: as+`implementation_version` (the `pipelex-api` distribution) and as `runtime_version` (the `pipelex` it runs).+`make docker-smoke` runs this against the container it starts.++It needs nothing but the standard library, so it runs on a bare runner without the member's environment.++Usage:+ python scripts/check_served_version.py http://127.0.0.1:18081 --version 0.71.0+"""++import argparse+import json+import sys+import urllib.error+import urllib.request++_VERSION_FIELDS = ("implementation_version", "runtime_version")+++def main() -> int:+ parser = argparse.ArgumentParser(description="Check the versions a running pipelex-api server reports.")+ parser.add_argument("base_url", help="The server's base URL, without the /v1 prefix")+ parser.add_argument("--version", required=True, help="The version both fields must report")+ args = parser.parse_args()++ version_url = f"{args.base_url.rstrip('/')}/v1/version"+ try:+ with urllib.request.urlopen(version_url, timeout=10) as response: # noqa: S310 - the URL is the smoke container's own+ version_info = json.load(response)+ except (urllib.error.URLError, TimeoutError, json.JSONDecodeError) as exc:+ print(f"Served-version check FAILED: cannot read {version_url}: {exc}")+ return 1+ print(f"{version_url}: {json.dumps(version_info)}")+ mismatches = {field: version_info.get(field) for field in _VERSION_FIELDS if version_info.get(field) != args.version}+ if mismatches:+ print(f"Served-version check FAILED: expected {args.version} for {', '.join(_VERSION_FIELDS)}, got {mismatches}.")+ return 1+ print(f"The server reports pipelex-api {args.version} running pipelex {args.version}.")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/api/scripts/export_openapi.py b/api/scripts/export_openapi.pyindex 777dd66f5..7627c769e 100644--- a/api/scripts/export_openapi.py+++ b/api/scripts/export_openapi.py@@ -1,15 +1,14 @@"""Export the FastAPI-generated OpenAPI schema to the committed YAML artifact.-The committed file (`docs/openapi/pipelex-api.openapi.yaml`) is the layer-2-contract of the MTHDS Protocol nesting (MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex-hosted API): the five protocol routes are tagged `x-mthds-protocol: true`, the-build tooling extensions ride alongside, and the non-contract storage routes-(`/upload`, `/resolve-storage-url`) are documented as such in their-descriptions.+The committed file (`docs/api-server/openapi/pipelex-api.openapi.yaml` at the repository root,+published on the pipelex docs site beside the server's pages) is the layer-2 contract of the MTHDS+Protocol nesting (MTHDS Protocol ⊂ Pipelex API ⊂ Pipelex hosted API): the five protocol routes are+tagged `x-mthds-protocol: true`, the build tooling extensions ride alongside, and the non-contract+storage routes (`/upload`, `/resolve-storage-url`) are documented as such in their descriptions.Usage:- python scripts/export_openapi.py docs/openapi/pipelex-api.openapi.yaml- python scripts/export_openapi.py --check docs/openapi/pipelex-api.openapi.yaml+ python scripts/export_openapi.py ../docs/api-server/openapi/pipelex-api.openapi.yaml+ python scripts/export_openapi.py --check ../docs/api-server/openapi/pipelex-api.openapi.yaml`--check` exits non-zero when the committed artifact drifts from the schema theapp currently generates — `make openapi-check`, which the gate (`make agent-check`, here or at thediff --git a/api/tests/unit/test_check_lockstep_pin.py b/api/tests/unit/test_check_lockstep_pin.pynew file mode 100644index 000000000..02550c3ba--- /dev/null+++ b/api/tests/unit/test_check_lockstep_pin.py@@ -0,0 +1,118 @@+"""`scripts/check_lockstep_pin.py` refuses a `pipelex-api` distribution that does not pin its pipelex exactly.++The release workflow runs it on the built wheel and sdist before uploading them, so it is the last thing between a+missing or wrong pin and PyPI. These tests build small distributions by hand, one defect at a time, and check that+each is reported. The script is a standalone entry point, not a package module, so it is loaded from its path.+"""++import importlib.util+import io+import tarfile+import zipfile+from pathlib import Path+from types import ModuleType++import pytest++_MEMBER_ROOT = Path(__file__).resolve().parents[2]+_SCRIPT_PATH = _MEMBER_ROOT / "scripts" / "check_lockstep_pin.py"+++def _load_script() -> ModuleType:+ spec = importlib.util.spec_from_file_location("check_lockstep_pin", _SCRIPT_PATH)+ assert spec is not None+ assert spec.loader is not None+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)+ return module+++check_lockstep_pin = _load_script()++VERSION = "1.2.3"+PINNED_REQUIREMENT = "pipelex[anthropic,bedrock,fal,google,google-genai,mistralai]==1.2.3"+CONFIGURED_REQUIREMENT = "pipelex[mistralai,anthropic,google,google-genai,bedrock,fal]"+++def _metadata(*, version: str = VERSION, requirements: list[str]) -> str:+ lines = ["Metadata-Version: 2.4", "Name: pipelex-api", f"Version: {version}"]+ lines.extend(f"Requires-Dist: {requirement}" for requirement in requirements)+ return "\n".join(lines) + "\n"+++def _write_wheel(*, dist_dir: Path, version: str, metadata_text: str) -> None:+ with zipfile.ZipFile(dist_dir / f"pipelex_api-{version}-py3-none-any.whl", "w") as wheel:+ wheel.writestr(f"pipelex_api-{version}.dist-info/METADATA", metadata_text)+ wheel.writestr("pipelex_api/__init__.py", "")+++def _write_sdist(*, dist_dir: Path, version: str, metadata_text: str) -> None:+ payload = metadata_text.encode("utf-8")+ member = tarfile.TarInfo(name=f"pipelex_api-{version}/PKG-INFO")+ member.size = len(payload)+ with tarfile.open(dist_dir / f"pipelex_api-{version}.tar.gz", "w:gz") as sdist:+ sdist.addfile(member, io.BytesIO(payload))+++def _write_distributions(*, dist_dir: Path, version: str = VERSION, requirements: list[str]) -> None:+ metadata_text = _metadata(version=version, requirements=requirements)+ _write_wheel(dist_dir=dist_dir, version=version, metadata_text=metadata_text)+ _write_sdist(dist_dir=dist_dir, version=version, metadata_text=metadata_text)+++def _check(dist_dir: Path, version: str = VERSION) -> list[str]:+ lockstep_requirements = [check_lockstep_pin.parse_requirement(CONFIGURED_REQUIREMENT)]+ problems: list[str] = check_lockstep_pin.check_distributions(+ dist_dir=dist_dir,+ version=version,+ lockstep_requirements=lockstep_requirements,+ )+ return problems+++class TestCheckLockstepPin:+ def test_exact_pin_in_wheel_and_sdist_passes(self, tmp_path: Path) -> None:+ _write_distributions(+ dist_dir=tmp_path,+ requirements=["fastapi>=0.118.0", PINNED_REQUIREMENT, "pytest>=8; extra == 'dev'"],+ )+ assert _check(tmp_path) == []++ @pytest.mark.parametrize(+ ("requirement", "expected_fragment"),+ [+ ("pipelex[anthropic,bedrock,fal,google,google-genai,mistralai]", "with `no version`, expected `==1.2.3`"),+ ("pipelex[anthropic,bedrock,fal,google,google-genai,mistralai]>=1.2.3", "with `>=1.2.3`, expected `==1.2.3`"),+ ("pipelex[anthropic,bedrock,fal,google,google-genai,mistralai]==1.2.2", "with `==1.2.2`, expected `==1.2.3`"),+ ("pipelex[anthropic,google]==1.2.3", "expected ['anthropic', 'bedrock', 'fal', 'google', 'google-genai', 'mistralai']"),+ ],+ )+ def test_a_wrong_pin_is_reported_for_both_distributions(self, tmp_path: Path, requirement: str, expected_fragment: str) -> None:+ _write_distributions(dist_dir=tmp_path, requirements=["fastapi>=0.118.0", requirement])+ problems = _check(tmp_path)+ assert len(problems) == 2+ assert all(expected_fragment in problem for problem in problems)++ def test_a_requirement_behind_a_marker_is_not_the_pin(self, tmp_path: Path) -> None:+ _write_distributions(dist_dir=tmp_path, requirements=[f"{PINNED_REQUIREMENT}; extra == 'dev'"])+ problems = _check(tmp_path)+ assert problems == [+ "pipelex_api-1.2.3-py3-none-any.whl carries 0 unconditional `Requires-Dist` for `pipelex`, expected exactly one.",+ "pipelex_api-1.2.3.tar.gz carries 0 unconditional `Requires-Dist` for `pipelex`, expected exactly one.",+ ]++ def test_distributions_of_another_version_are_refused(self, tmp_path: Path) -> None:+ _write_distributions(dist_dir=tmp_path, requirements=[PINNED_REQUIREMENT])+ problems = _check(tmp_path, version="1.2.4")+ assert "pipelex_api-1.2.3-py3-none-any.whl is not a 1.2.4 distribution: expected pipelex_api-1.2.4-py3-none-any.whl." in problems+ assert "pipelex_api-1.2.3.tar.gz declares `Version: 1.2.3`, expected `1.2.4`." in problems++ def test_a_missing_sdist_is_refused(self, tmp_path: Path) -> None:+ _write_wheel(dist_dir=tmp_path, version=VERSION, metadata_text=_metadata(requirements=[PINNED_REQUIREMENT]))+ problems = _check(tmp_path)+ assert problems == [f"Expected exactly one `pipelex-api` wheel and one sdist in {tmp_path}, found: pipelex_api-1.2.3-py3-none-any.whl."]++ def test_the_configured_lockstep_requirement_names_pipelex_without_a_version(self) -> None:+ lockstep_requirements = check_lockstep_pin.read_lockstep_requirements()+ assert [requirement.name for requirement in lockstep_requirements] == ["pipelex"]+ assert all(requirement.specifier == "" and requirement.marker == "" for requirement in lockstep_requirements)diff --git a/api/tests/unit/test_openapi_contract.py b/api/tests/unit/test_openapi_contract.pyindex 141153fe1..73ac88bd8 100644--- a/api/tests/unit/test_openapi_contract.py+++ b/api/tests/unit/test_openapi_contract.py@@ -1,7 +1,7 @@"""The published error contract — every `/v1` operation documents RFC 7807, with real schemas.-Guards what the committed artifact (`docs/openapi/pipelex-api.openapi.yaml`, regenerated from-this same `fastapi_app.openapi()`) tells a client about failures:+Guards what the committed artifact (`docs/api-server/openapi/pipelex-api.openapi.yaml` at the repository root,+regenerated from this same `fastapi_app.openapi()`) tells a client about failures:- Every failure this server can emit is an `application/problem+json` problem document(`pipelex_api.exception_handlers`), so every documented 4xx/5xx must say so — with a `$ref` to thediff --git a/docs/api-server/changelog.md b/docs/api-server/changelog.mdnew file mode 100644index 000000000..53b8b0dec--- /dev/null+++ b/docs/api-server/changelog.md@@ -0,0 +1,6 @@+---+title: API Server Changelog to v0.33.2+description: "The release history of the Pipelex API server up to v0.33.2, its last release before it joined the pipelex repository. Later changes are in the Pipelex changelog."+---++--8<-- "api/CHANGELOG.md"diff --git a/api/docs/codegen.md b/docs/api-server/codegen.mdsimilarity index 100%rename from api/docs/codegen.mdrename to docs/api-server/codegen.mddiff --git a/api/docs/configuration.md b/docs/api-server/configuration.mdsimilarity index 100%rename from api/docs/configuration.mdrename to docs/api-server/configuration.mddiff --git a/api/docs/error-responses.md b/docs/api-server/error-responses.mdsimilarity index 100%rename from api/docs/error-responses.mdrename to docs/api-server/error-responses.mddiff --git a/api/docs/index.md b/docs/api-server/index.mdsimilarity index 89%rename from api/docs/index.mdrename to docs/api-server/index.mdindex c0708d793..f14d64335 100644--- a/api/docs/index.md+++ b/docs/api-server/index.md@@ -1,9 +1,14 @@-# Pipelex API Documentation+---+title: Pipelex API Server+description: "The Pipelex API server: the self-hostable FastAPI runner published as the pipelex/pipelex-api Docker image and the pipelex-api package, and the reference implementation of the MTHDS Protocol."+----!!! important "This server lives in [`Pipelex/pipelex`](https://github.com/Pipelex/pipelex), as the repository's `api/` directory"- It moved there from `Pipelex/pipelex-api` after v0.33.2 and is released together with the pipelex library, under the library's version number, so the tag of the `pipelex/pipelex-api` image that follows 0.33.2 is a pipelex version. The image keeps its name, its port and its `/root/.pipelex` configuration mount. Please open issues on `Pipelex/pipelex`.+# Pipelex API Server-Welcome to the Pipelex API documentation. The API provides programmatic access to the Pipelex system.+The Pipelex API server runs MTHDS methods over HTTP on infrastructure you host yourself. It is published as the [`pipelex/pipelex-api`](https://hub.docker.com/r/pipelex/pipelex-api) Docker image and as the `pipelex-api` package on PyPI, and its source is the [`api/` directory](https://github.com/Pipelex/pipelex/tree/main/api) of the pipelex repository.++!!! note "Released with pipelex, under pipelex's version"+ Every pipelex release also ships the server: the `pipelex-api` package pins the `pipelex` of the same version, and the image tag is that version too, so `pipelex/pipelex-api:X.Y.Z` runs pipelex X.Y.Z. The server was released on its own, from the `Pipelex/pipelex-api` repository, until v0.33.2; its image tag then jumped to the pipelex version that first shipped it. Its history up to then is the [server's changelog to v0.33.2](changelog.md), and its changes since are in the [Pipelex changelog](../changelog.md).## The three-layer contract@@ -45,7 +50,7 @@ docker run --name pipelex-api -p 8081:8081 \pipelex/pipelex-api:latest```-To require authentication on the API itself, add `-e AUTH_MODE=api_key -e API_KEY=your-secret` (or `AUTH_MODE=jwt` + `JWT_SECRET_KEY`). The full set of accepted env vars is documented in [Configuration](configuration.md) and `.env.example`.+To require authentication on the API itself, add `-e AUTH_MODE=api_key -e API_KEY=your-secret` (or `AUTH_MODE=jwt` + `JWT_SECRET_KEY`). The full set of accepted env vars is documented in [Configuration](configuration.md) and in the server's [`.env.example`](https://github.com/Pipelex/pipelex/blob/main/api/.env.example).If you'd rather keep config out of your shell history, use `--env-file .env` or a `docker-compose.yml` instead — see [Configuration → Setting env vars in Docker](configuration.md#setting-env-vars-in-docker) for both patterns.diff --git a/api/docs/logging.md b/docs/api-server/logging.mdsimilarity index 98%rename from api/docs/logging.mdrename to docs/api-server/logging.mdindex fdc06ea22..2d71fe4a1 100644--- a/api/docs/logging.md+++ b/docs/api-server/logging.md@@ -59,7 +59,7 @@ Disposition follows the HTTP status, not the error domain: a `4xx` is a caller m## Configuration-The keys live in `[runtime.log]` of the `.pipelex/pipelex.toml` this repository ships, which the image copies to `/root/.pipelex/`:+The keys live in `[runtime.log]` of the server's own `api/.pipelex/pipelex.toml`, which the image copies to `/root/.pipelex/`:```toml[runtime.log]diff --git a/api/docs/mthds-tools.md b/docs/api-server/mthds-tools.mdsimilarity index 100%rename from api/docs/mthds-tools.mdrename to docs/api-server/mthds-tools.mddiff --git a/api/docs/openapi/pipelex-api.openapi.yaml b/docs/api-server/openapi/pipelex-api.openapi.yamlsimilarity index 100%rename from api/docs/openapi/pipelex-api.openapi.yamlrename to docs/api-server/openapi/pipelex-api.openapi.yamldiff --git a/api/docs/pipe-builder.md b/docs/api-server/pipe-builder.mdsimilarity index 100%rename from api/docs/pipe-builder.mdrename to docs/api-server/pipe-builder.mddiff --git a/api/docs/pipe-io.md b/docs/api-server/pipe-io.mdsimilarity index 100%rename from api/docs/pipe-io.mdrename to docs/api-server/pipe-io.mddiff --git a/api/docs/pipe-run.md b/docs/api-server/pipe-run.mdsimilarity index 100%rename from api/docs/pipe-run.mdrename to docs/api-server/pipe-run.mddiff --git a/api/docs/pipe-validate.md b/docs/api-server/pipe-validate.mdsimilarity index 100%rename from api/docs/pipe-validate.mdrename to docs/api-server/pipe-validate.mddiff --git a/api/docs/storage-transport.md b/docs/api-server/storage-transport.mdsimilarity index 100%rename from api/docs/storage-transport.mdrename to docs/api-server/storage-transport.mddiff --git a/docs/building-methods/pipes/executing-pipelines.md b/docs/building-methods/pipes/executing-pipelines.mdindex 8485d5ba8..0725db411 100644--- a/docs/building-methods/pipes/executing-pipelines.md+++ b/docs/building-methods/pipes/executing-pipelines.md@@ -216,7 +216,7 @@ pipe_output = response.pipe_output## Using the Pipelex API-Pipelex has a REST API for executing pipelines. See more about it [here](https://pipelex.github.io/pipelex-api/).+The [Pipelex API server](../../api-server/index.md) runs pipelines over HTTP on infrastructure you host yourself: it is published as the `pipelex/pipelex-api` Docker image, and [Pipe Run](../../api-server/pipe-run.md) documents its `/v1/execute` and `/v1/start` routes.---diff --git a/docs/contribute/api-server.md b/docs/contribute/api-server.mdindex ed18e81e5..44774ba10 100644--- a/docs/contribute/api-server.md+++ b/docs/contribute/api-server.md@@ -1,11 +1,11 @@---title: "The API Server"-description: "The Pipelex API server lives in this repository as the api/ workspace member: how it depends on the library, its environment, the gate that covers it, its vendored inference tree and its image."+description: "The Pipelex API server lives in this repository as the api/ workspace member: how it depends on the library, its environment, the gate and CI that cover it, its vendored inference tree, its image and how a release ships it."---# The API server-The Pipelex API server, the FastAPI runner published as the `pipelex/pipelex-api` image, lives in this repository as the `api/` directory. It is the reference implementation of the MTHDS Protocol, and the base the hosted runner composes. Its own guide is `api/CLAUDE.md`, and its reference documentation is under `api/docs/`.+The Pipelex API server, the FastAPI runner published as the `pipelex/pipelex-api` image and the `pipelex-api` package, lives in this repository as the `api/` directory. It is the reference implementation of the MTHDS Protocol, and the base the hosted runner composes. Its own guide is `api/CLAUDE.md`, and its reference documentation is the [API Server](../api-server/index.md) section of this site, whose pages live in `docs/api-server/`.## A workspace member@@ -18,7 +18,9 @@ members = ["api"]The member depends on the library through the workspace (`pipelex = { workspace = true }` in its `[tool.uv.sources]`), so the server always runs the library of the same commit, and the repository keeps a single `uv.lock` at its root. The `pipelex` wheel and sdist are unaffected: both restrict themselves to `packages = ["pipelex"]`.-The two are released together, under one version number. The member declares its version dynamic and hatch reads it from the root `pyproject.toml` (`[tool.hatch.version]` in `api/pyproject.toml`), so a release bumps one file, and the member's `[tool.uv] cache-keys` name the root file so that uv notices the bump. Its dependency on `pipelex` names no version, because uv ignores a version specifier on a workspace source and a pin there would only go stale.+The two are released together, under one version number. The member declares its version dynamic and hatch reads it from the root `pyproject.toml` (`[tool.hatch.version]` in `api/pyproject.toml`), so a release bumps one file, and the member's `[tool.uv] cache-keys` name the root file so that uv notices the bump.++Its dependencies are dynamic too. They are written in `[tool.hatch.metadata.hooks.custom]` of `api/pyproject.toml` rather than in `[project]`, and `api/hatch_build.py` writes them into the distribution's metadata: each requirement of `lockstep-dependencies`, which is `pipelex` with the server's extras, pinned to `==` the release's own version, then the other `dependencies` as written. In the workspace uv ignores a version specifier on a workspace source, so the pin changes nothing locally and never goes stale; in a published `pipelex-api` it is what makes an install take exactly the pipelex the server was released with. A server dependency is therefore added or changed in that table by hand, and the root's `uv lock` picks it up: `uv add` would write a static `[project] dependencies` list, which hatch refuses beside the dynamic one.## Its own environment@@ -26,11 +28,22 @@ The server is tested with exactly the dependencies it declares, which are not th## The gate covers it-The root `make agent-check` ends with `make api-agent-check`, which runs `make -C api agent-check`: the member's install, ruff, pyright and mypy under its own configuration, then two drift checks. The root `make agent-test` ends with `make api-agent-test`, the server's unit tests. A library change that breaks the server therefore fails the gate in the change that makes it, and so does one that moves the server's wire or its inference tree. pipelex's pull-request workflows run the library's checks only, so it is this gate, run before pushing, that catches them:+The root `make agent-check` ends with `make api-agent-check`, which runs `make -C api agent-check`: the member's install, ruff, pyright and mypy under its own configuration, then two drift checks. The root `make agent-test` ends with `make api-agent-test`, the server's unit tests. A library change that breaks the server therefore fails the gate in the change that makes it, and so does one that moves the server's wire or its inference tree:-- **`make -C api openapi-check`** compares the committed OpenAPI artifact, `api/docs/openapi/pipelex-api.openapi.yaml`, with what the app generates. pipelex's own models ride the server's wire (the run result, the bundle blueprint, the validation error vocabulary), so a library change can move the artifact with no change to the server's code. Regenerate it with `make -C api openapi-export` and read the diff: it is what the change did to the server's callers.+- **`make -C api openapi-check`** compares the committed OpenAPI artifact, `docs/api-server/openapi/pipelex-api.openapi.yaml`, with what the app generates. It sits among the server's pages, so the docs site publishes it beside the pages that link to it. pipelex's own models ride the server's wire (the run result, the bundle blueprint, the validation error vocabulary), so a library change can move the artifact with no change to the server's code. Regenerate it with `make -C api openapi-export` and read the diff: it is what the change did to the server's callers.- **`make -C api kit-check`** compares the server's vendored inference tree with the kit.+## Pull-request CI covers it++Every pull request runs the server's checks, with no path filter, because a change to the library alone can break the server:++- **`Lint (api)`**, in `lint-check.yml`, runs the member's ruff, pyright and mypy, `openapi-check` and `kit-check` in its own environment.+- **`Tests (api)`**, in `tests-check.yml`, runs its unit tests.+- **`Tests (packages)`** builds the `pipelex` and `pipelex-api` wheels and sdists as a release would, checks the exact pin on `pipelex` in the `pipelex-api` metadata with `api/scripts/check_lockstep_pin.py`, installs the two wheels together under the lock's versions and imports the server from them.+- **`Tests (api image)`** builds the image from the repository root and smoke-tests it with `make -C api docker-smoke`, which boots it, waits for `/health` and checks that `/v1/version` reports this version for both the server and the pipelex it runs.++They block through the `Lint (all)` and `Tests (all)` aggregates. Those run the floor Python version only, as they do for the library; on a pull request into `main`, the pre-main gates add the server to every supported version, in each `Lint fresh (pyX)` leg and in the `Tests full (api, pyX)` jobs.+The root ruff configuration excludes `api/`, which the member's gate lints under its own. The root plxt configuration formats and lints the member's TOML and MTHDS files like any other, except its vendored inference tree, which the sync keeps byte-identical to the kit, and its deliberately invalid sample bundles.## The vendored inference tree@@ -47,4 +60,16 @@ The image is built with the repository root as its context, so it installs the pdocker build -f api/Dockerfile .```-`make -C api docker-build` runs exactly that. The root `.dockerignore` keeps the context down to the files the build reads. It is an allowlist, and it sits at the root rather than beside the Dockerfile because the legacy builder reads no other ignore file, and without one it would copy local override files, which can hold credentials, into the image.+`make -C api docker-build` runs exactly that, and `make -C api docker-smoke` boots what it built and checks the versions it reports. The root `.dockerignore` keeps the context down to the files the build reads. It is an allowlist, and it sits at the root rather than beside the Dockerfile because the legacy builder reads no other ignore file, and without one it would copy local override files, which can hold credentials, into the image.++## How a release ships it++A pipelex release publishes three artifacts under its one version, all from `.github/workflows/publish-pypi.yml`, which fires when the release pull request merges into `main`:++1. **`pipelex` on PyPI**, as before: `build`, then `publish-to-pypi`, then `github-release`, which tags the release commit and refuses a commit an existing tag does not name.+2. **`pipelex-api` on PyPI**: `build-api` builds the server's wheel and sdist and refuses them unless `check_lockstep_pin.py` finds `pipelex[...]==X.Y.Z` in both. `publish-to-pypi` waits for it, so a server that does not build stops the release before `pipelex` is uploaded, while the version can still ship from a fixed commit. Then `publish-api-to-pypi` uploads them through trusted publishing in the same `pypi` environment. It waits for `github-release`, so the `pipelex` wheel they pin is on PyPI and the tag guard has accepted the commit they were built from before a file reaches PyPI, where it can never be replaced.+3. **`pipelex/pipelex-api:X.Y.Z` on Docker Hub**: once both distributions are on PyPI, `publish-docker-hub` calls `.github/workflows/publish-docker-hub.yml` with the version. It checks out the commit the `vX.Y.Z` tag names, runs `make -C api deploy-docker-hub`, which builds the image, smoke-tests it and pushes it, confirms the tags through Docker Hub's API, and pushes `api/README.md` as the Docker Hub overview. `latest` moves only when the version is the newest stable release, so a pre-release, or a retry of an older release, leaves it where it is.++A failed publish is retried with "Re-run failed jobs" on the release run, which re-runs only what failed. A version Docker Hub already serves is never built again, because a rebuild would be a different image under the same tag: a retry that finds it there only moves `latest`, when the version is the newest stable release, and confirms the tags. The image can also be published on its own with `gh workflow run publish-docker-hub.yml --ref <branch> -f version=X.Y.Z`, which helps when the fix is a change to that workflow file, which a re-run would not pick up: dispatched from the branch carrying the fix, it still builds the commit the tag names.++What the release needs configured on GitHub and PyPI: the `DOCKER_HUB_TOKEN` secret on the repository, a token for the `pipelex` Docker Hub account that can push the image and, for the overview, has the read/write/delete scope; and a PyPI trusted publisher for the `pipelex-api` project with owner `Pipelex`, repository `pipelex`, workflow `publish-pypi.yml` and environment `pypi`, the same as `pipelex`'s.diff --git a/docs/under-the-hood/codegen-projections.md b/docs/under-the-hood/codegen-projections.mdindex f0f67bb43..a302aa50f 100644--- a/docs/under-the-hood/codegen-projections.md+++ b/docs/under-the-hood/codegen-projections.md@@ -167,7 +167,7 @@ The remaining findings are artifacts of *how* you lint, not of file content. Non## Serving the engine over HTTP-The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen`, and the re-pointed `/v1/build/*` — the route envelopes are documented in the server's `api/docs/codegen.md`). Two host-facing cores make that possible without any CLI plumbing:+The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen`, and the re-pointed `/v1/build/*` — the route envelopes are documented on the API server's [Resolve & Codegen](../api-server/codegen.md) page). Two host-facing cores make that possible without any CLI plumbing:- **`pipelex.pipeline.resolve_bundle.resolve_crate_from_contents`** resolves **in-memory** MTHDS contents (strings, with optional per-content sources) into the normalized crate. It mirrors `validate_bundle`'s in-memory arm — the same `translate_to_validate_bundle_error` cascade, so an invalid library raises the one shared `ValidateBundleError` and a resolve verdict cannot drift from a validate verdict — and the same **loaded-on-success contract**: the library is left loaded and current for the host to read live pipes from, and the host owns its teardown. Resolution is static (no dry-run sweep), matching `pipelex resolve`.- **`build_stamped_projection`** (above) gives the host the stamped artifact set plus the lock as pure content. A client that writes the served files and lock verbatim reproduces a local run byte-for-byte — the offline `codegen check` passes on the written tree exactly as it would locally. There is deliberately no server-side check route: the check is offline by design.diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.mdindex 9940d22ae..dceab9a92 100644--- a/docs/under-the-hood/error-model.md+++ b/docs/under-the-hood/error-model.md@@ -141,7 +141,7 @@ The **fix planner** (`pipelex/pipeline/fixes/planner.py`) translates enriched tyApplying fixes is the runtime's job too: the **applier** (`pipelex/pipeline/fixes/applier.py`) mutates a tomlkit DOM in place per op (guarded — an op whose target table is absent is skipped and reported, never raised) and then reflows the whole file to canonical MTHDS style, and the **convergence loop** (`pipelex/pipeline/fixes/fix_loop.py`) runs validate → apply SAFE fixes → re-validate to a fixed point, reporting non-convergence loudly. The user-facing surface is [`pipelex fix bundle`](../tools/cli/fix.md).-On the hosted API the same payload rides the wire verbatim as `validation_errors[].suggested_fix`; how it appears in HTTP error responses is documented on the API side, in the server's `api/docs/error-responses.md` → "Suggested fixes".+On the hosted API the same payload rides the wire verbatim as `validation_errors[].suggested_fix`; how it appears in HTTP error responses is documented on the API side, in the API server's [Error Responses](../api-server/error-responses.md#suggested-fixes) page.---diff --git a/mkdocs.yml b/mkdocs.ymlindex 442d46857..37002be6b 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -375,6 +375,17 @@ plugins:- contribute/api-server.md: "The API Server"- migration-ledger.md: "Migration Ledger"- changelog.md: "Changelog"+ API Server:+ - api-server/index.md: "Pipelex API Server Overview"+ - api-server/configuration.md: "API Server Configuration"+ - api-server/pipe-run.md: "API Server Pipe Run"+ - api-server/pipe-validate.md: "API Server Pipe Validate"+ - api-server/pipe-io.md: "API Server Pipe I/O"+ - api-server/codegen.md: "API Server Resolve & Codegen"+ - api-server/pipe-builder.md: "API Server Pipe Builder"+ - api-server/mthds-tools.md: "API Server MTHDS Tools"+ - api-server/error-responses.md: "API Server Error Responses"+ - api-server/logging.md: "API Server Logging"markdown_extensions:- meta@@ -617,4 +628,16 @@ nav:- Code of Conduct: CODE_OF_CONDUCT.md- License: license.md- Changelog: changelog.md- - API: https://pipelex.github.io/pipelex-api/+ - API Server:+ - Overview: api-server/index.md+ - Configuration: api-server/configuration.md+ - Pipe Run: api-server/pipe-run.md+ - Pipe Validate: api-server/pipe-validate.md+ - Pipe I/O: api-server/pipe-io.md+ - Resolve & Codegen: api-server/codegen.md+ - Pipe Builder: api-server/pipe-builder.md+ - MTHDS Tools: api-server/mthds-tools.md+ - Error Responses: api-server/error-responses.md+ - Logging: api-server/logging.md+ - Storage Transport (Removed): api-server/storage-transport.md+ - Changelog to v0.33.2: api-server/changelog.mddiff --git a/uv.lock b/uv.lockindex cae5d407c..e6bc44f13 100644--- a/uv.lock+++ b/uv.lock@@ -3789,22 +3789,12 @@ dev = [{ name = "types-requests" },{ name = "types-toml" },]-docs = [- { name = "mkdocs" },- { name = "mkdocs-glightbox" },- { name = "mkdocs-material" },- { name = "mkdocs-meta-manager" },-][package.metadata]requires-dist = [{ name = "boto3-stubs", marker = "extra == 'dev'", specifier = ">=1.35.24" },- { name = "fastapi", specifier = ">=0.118.0" },+ { name = "fastapi", specifier = ">=0.118.0,<0.142" },{ name = "httpx2", marker = "extra == 'dev'", specifier = ">=2.4.0" },- { name = "mkdocs", marker = "extra == 'docs'", specifier = ">=1.6.1" },- { name = "mkdocs-glightbox", marker = "extra == 'docs'", specifier = ">=0.4.0" },- { name = "mkdocs-material", marker = "extra == 'docs'", specifier = ">=9.7.7" },- { name = "mkdocs-meta-manager", marker = "extra == 'docs'", specifier = ">=1.1.0" },{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.11.2" },{ name = "pandas-stubs", marker = "extra == 'dev'", specifier = ">=2.2.3.241126" },{ name = "pipelex", extras = ["anthropic", "bedrock", "fal", "google", "google-genai", "mistralai"], editable = "." },@@ -3829,7 +3819,7 @@ requires-dist = [{ name = "types-toml", marker = "extra == 'dev'", specifier = ">=0.10.8.20240310" },{ name = "uvicorn", specifier = ">=0.37.0" },]-provides-extras = ["dev", "docs"]+provides-extras = ["dev"][[package]]name = "pipelex-tools"