Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Readme-byok · L-260918-9ce069 — docs stop describing the Gateway (#1228)
Thomas Hebrard62bf4e67a2d7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4913d2570..746b536e8 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,11 @@# Changelog+## [Unreleased]++### Changed++- **The documentation no longer describes the Pipelex Gateway**: **Configure AI Providers** offers your own provider keys or a local model, says that the shipped defaults run as they are on OpenAI or Azure OpenAI and must be repointed for any other provider, OpenRouter included, and points to the Quick Start for running on the hosted API; the MTHDS language tutorial no longer runs `pipelex login`. The telemetry, logging, feature and under-the-hood pages describe only the telemetry streams you configure yourself and the backends the runtime ships, without the Gateway stream, its backend or its plugin.+## [v0.75.0] - 2026-10-06### Addeddiff --git a/docs/api-server/logging.md b/docs/api-server/logging.mdindex 327e941be..3e11aceed 100644--- a/docs/api-server/logging.md+++ b/docs/api-server/logging.md@@ -26,7 +26,7 @@ These keys come from the sink itself and are on every line written by the proces| `message` | The human-readable summary || `exception` | The traceback, when the record carries one || `trace_id`, `span_id`, `trace_flags` | The process's current OpenTelemetry span, in lowercase hex, when one names a trace. This server registers no tracer and instruments nothing, so as shipped no line carries them; they appear only when the deployment runs it under OpenTelemetry instrumentation of its own that makes a span current |-| `pipelex.trace_id`, `pipelex.span_id` | The Pipelex span a runtime line was logged in, the pipe's or the LLM call's, in lowercase hex. Present on the runtime's lines inside a live run the runtime traces, which it does whenever the Pipelex Gateway's telemetry is on or AI span tracing is enabled on your own PostHog; never on the server's own lines, an error line included |+| `pipelex.trace_id`, `pipelex.span_id` | The Pipelex span a runtime line was logged in, the pipe's or the LLM call's, in lowercase hex. Present on the runtime's lines inside a live run the runtime traces, which it does when AI span tracing is enabled on your own PostHog; never on the server's own lines, an error line included |The two pairs name different traces. Pipelex's spans belong to a trace of their own, which only Pipelex's exporters receive, so if you export them to a backend of your own, such as Langfuse, an OTLP collector or your PostHog, join a line to them on `pipelex.trace_id` and `pipelex.span_id`. To gather a run's lines whether or not anything traces, use `request_id` or `pipeline_run_id`. How each sink writes both pairs is in the Pipelex documentation's [trace context](https://docs.pipelex.com/0.69.0/tools/logging/#the-trace-context) section.diff --git a/docs/configuration/config-technical/inference-config.md b/docs/configuration/config-technical/inference-config.mdindex 840fc88fa..9ee753feb 100644--- a/docs/configuration/config-technical/inference-config.md+++ b/docs/configuration/config-technical/inference-config.md@@ -30,7 +30,7 @@ transport_max_retries = 2- `transport_max_retries` (int, `0`–`10`, default `2`): the number of retries attempted **on top of** the initial request when a transport-level failure occurs — a connection error, or an HTTP `408` / `409` / `429` / `5xx` response. A value of `2` therefore allows up to 3 attempts total. Retries honor a `Retry-After` response header when present.-This is "Tier 1" of the retry model. It is wired uniformly into every inference SDK client factory — Anthropic, OpenAI / Azure OpenAI, the Pipelex Gateway clients, Mistral, and Google — as well as the raw-`httpx` Azure image-generation path, so the retry posture is a deliberate, uniform policy rather than a per-provider SDK default.+This is "Tier 1" of the retry model. It is wired uniformly into every inference SDK client factory — Anthropic, OpenAI / Azure OpenAI and the other backends reached through the OpenAI SDK, Portkey, Mistral, Google, and TypeSafe — as well as the raw-`httpx` Azure image-generation path, so the retry posture is a deliberate, uniform policy rather than a per-provider SDK default.It is distinct from `inference.llm.schema_reask_max_attempts`, which is `instructor`'s schema re-ask count for structured-output validation failures — a different concern.diff --git a/docs/contribute/drift-contracts.md b/docs/contribute/drift-contracts.mdindex 62c076c45..c285cd4d5 100644--- a/docs/contribute/drift-contracts.md+++ b/docs/contribute/drift-contracts.md@@ -8,11 +8,11 @@ Not every code↔doc relationship deserves a review contract. This repo distingu| Tier | Mechanism | Judgment needed ||---|---|---|-| **Derived** | Regenerate and diff: `check-config-sync`, `check-mthds-schema`, `check-gateway-models`, `generate-error-pages`, `generate-error-identity` | None — the output is a pure function of its sources |+| **Derived** | Regenerate and diff: `check-config-sync`, `check-mthds-schema`, `generate-error-pages`, `generate-error-identity` | None — the output is a pure function of its sources || **Linkage** | Referential integrity of declared cross-references (the spec ↔ verifying-test back-link pair) | None — links either resolve or they don't || **Review** | Drift contracts: "this changed → that must be looked at, and the look recorded" | Yes — someone must judge whether the target still holds |-The standing rule: **anything mechanizable becomes a derived check.** If a review contract keeps opening and the review is always the same mechanical comparison, delete the contract and write a derived check instead — it needs no judgment and can't be rubber-stamped. The best review contract is one you eventually mechanize out of existence. Corollary: generated artifacts (`docs/errors/`, `derived/`, `tests/data/errors/error_identity.txt`, gateway model docs, the migration goldens) must never appear in a contract's `review` list — reviewing generator *output* is how you end up editing generated files, and a generated artifact's staleness is already somebody's derived check.+The standing rule: **anything mechanizable becomes a derived check.** If a review contract keeps opening and the review is always the same mechanical comparison, delete the contract and write a derived check instead — it needs no judgment and can't be rubber-stamped. The best review contract is one you eventually mechanize out of existence. Corollary: generated artifacts (`docs/errors/`, `derived/`, `tests/data/errors/error_identity.txt`, the migration goldens) must never appear in a contract's `review` list — reviewing generator *output* is how you end up editing generated files, and a generated artifact's staleness is already somebody's derived check.**A review target does not have to be prose.** Most are docs, because most staleness is a sentence that stopped being true — but the obligation is "look at this and judge", and what has to be judged is sometimes source. `config-docs` carries the migration ledgers and the two modules holding the four validators that narrow a configuration value domain: a narrowing expressed in a validator is invisible to the fingerprint projection the coverage gate reads, so no derived check can ever demand the ledger entry it owes, and the review list is the only place that obligation can land. The bar is the same as for a doc — the target must be something a person reads and judges, never something a command regenerates.diff --git a/docs/contribute/hub-layering.md b/docs/contribute/hub-layering.mdindex 5ff4b1045..17a43a38a 100644--- a/docs/contribute/hub-layering.md+++ b/docs/contribute/hub-layering.md@@ -317,7 +317,7 @@ Named so this document stays honest.**The one an aggregate module hid.** Five built-in vendor adapters — `providers/{anthropic,bedrock,google,mistral}/*_list.py` — imported `MissingDependencyError` from `pipelex.exceptions` rather than from its definition site in `pipelex.system.exceptions`. `pipelex/exceptions.py` is the *public all-errors aggregate*: it re-exports every interpreter package's exceptions, so importing anything from it loads `pipeline.exceptions` → `validation_errors` → `fixes.planner` and more. Each of those five declared kernel-layer modules therefore loaded 15 interpreter modules. Neither gate could see it: the guard's rules only chase `interpreter_hub`, and the aggregate never reaches it; the closure test's `providers.builtins` entry point does not cover these modules either, because the vendor plugins import them *inside a function*, which hides them from the static import graph and from the aggregator's own closure at once. The fix is the repo's existing rule — *direct full-path imports everywhere*, i.e. import from where the symbol is defined, never from a convenience aggregate — plus one new entry point (`providers.anthropic.anthropic_list`) so the runtime-truth check now covers the shape. The generalizable lesson: **a module that re-exports across layers is a layer boundary with the sign filed off**, and a deferred import will hide it from every static gate you have. Since then the rule is mechanical rather than prose: `tests/unit/pipelex/test_kernel_layer_exceptions_aggregate_gate.py` AST-walks every package declared in `KERNEL_LAYER_PACKAGES` and fails on any import of `pipelex.exceptions` — module-level or function-local, absolute or relative — as well as on any bare string naming it, since an `import_module` argument or a patch target loads the aggregate through a path no import node records. The next aggregate reference in the kernel layer is caught by CI instead of by review. The aggregate itself remains a legitimate public surface for callers *outside* the kernel layer; the ban is scoped to the layer whose closure it would silently widen.- **`cogt` names four specific vendors, deliberately.** `cogt/config_cogt.py` imports `AnthropicConfig`, `GoogleConfig`, `MistralConfig` and `OpenAIConfig` from their `pipelex/providers/` packages. Both sides are kernel-layer, so this breaches no hub rule — it is a placement inversion: the vendor-neutral engine names four vendors. It is accepted rather than fixed, because the alternative is worse. The main config model is statically typed end-to-end (`system/configuration/configs.py` and the `pipelex.toml` files are structurally synced, and boot fails loud when they disagree), so making vendor config sections plugin-contributed would trade compile-time typing for a dynamic registry — a real loss to buy a cosmetic win. The edge is one-way and shallow: a config class, not a worker.-- **The img-gen one that stopped being an inversion.** `cogt/img_gen/img_gen_args_factory.py` used to import two vendor img-gen factories at module level, for taxonomy and geometry helpers. It was never a registry problem: dispatch is keyed on `AspectRatioTaxonomy`, a `cogt`-owned enum that names model families and no providers, and no family is served by a single adapter — the `openai` and `azure_openai` decks both ship GPT Image models, and the gateway worker resolves Gemini geometry through the same mapping the native Google worker uses — so there was no provider key to dispatch on, and a registrar slot would have bought a plugin-API bump for two import sites. Fixed by placement instead: the helpers are now `cogt/img_gen/img_gen_gemini_mapping.py` and `cogt/img_gen/img_gen_gpt_mapping.py`, and the Google worker and the gateway factory import *inward* to them. One `cogt → specific vendor` edge of this kind remains — `model_backends/backend_factory.py` reaching for `VertexAIFactory` — already deferred inside a function, and it self-resolves when VertexAI support is removed.+- **The img-gen one that stopped being an inversion.** `cogt/img_gen/img_gen_args_factory.py` used to import two vendor img-gen factories at module level, for taxonomy and geometry helpers. It was never a registry problem: dispatch is keyed on `AspectRatioTaxonomy`, a `cogt`-owned enum that names model families and no providers, and no family is served by a single adapter — the `openai` and `azure_openai` decks both ship GPT Image models, and a former release's gateway worker resolved Gemini geometry through the same mapping the native Google worker uses — so there was no provider key to dispatch on, and a registrar slot would have bought a plugin-API bump for two import sites. Fixed by placement instead: the helpers are now `cogt/img_gen/img_gen_gemini_mapping.py` and `cogt/img_gen/img_gen_gpt_mapping.py`, and the Google worker and `img_gen_args_factory.py` import *inward* to them. One `cogt → specific vendor` edge of this kind remains — `model_backends/backend_factory.py` reaching for `VertexAIFactory` — already deferred inside a function, and it self-resolves when VertexAI support is removed.- **What pins the two bullets above.** The complete `cogt → pipelex.providers` list — the four config imports plus that one deferred factory import — is a golden set in `tests/unit/pipelex/cogt/test_cogt_dependency_boundaries.py`, together with the stronger fact that `pipelex/cogt/**` imports no vendor SDK at all. A sixth edge fails it, and nothing else in the repo would notice: both packages are kernel-layer, so the guard and the closure test are blind to an edge between them by construction. Golden sets rather than a rule, because these are *accepted* inversions — there is no predicate separating the sanctioned edges from a new one, and the point is to make the next one a diff a reviewer reads. Note the tempting generalization — *no vendor SDK anywhere in `pipelex/` outside `providers/`* — is **false**, and a test asserting it would be red on day one: `tools/pdf/pypdfium2_renderer.py`, `tools/storage/`, `tracing/` and `reporting/` import pypdfium2 and the AWS/GCP SDKs on purpose, because `providers/storage/` is a registration shim whose implementations live under `pipelex/tools/`. The scope that holds is `cogt`.- `plugins/pipe_func_executor_registry.py` is typed by a protocol from `pipe_operators/` and defers that import under `TYPE_CHECKING` precisely because `pipelex.config` imports it and the inference layer imports `pipelex.config` — a module-level import there would drag the interpreter back into every inference closure. Being type-only it loads nothing, so it breaches neither rule — the sanctioned deferral pattern. The placement inversion itself is unfixed.- **The one that stopped being an inversion.** `core/`'s Pipe-touching half — `pipe_abstract`, `pipe_blueprint`, `pipe_factory`, the renderers — imported the interpreter directly while sitting under a package the kernel layer wanted to declare. That was never a wrong *import* (it is a fact about what a pipe is), it was a wrong *address*, and it forced the declaration to enumerate core's data-model packages instead of naming `pipelex.core`. Fixed by placement: the modules are now `pipelex.pipe_machinery` and `pipelex.mthds_parsing`, and the six `pipelex.core.*` entries collapsed to one. Several of them still import `interpreter_hub` on purpose, to inject downward — that is the sanctioned direction and always was.diff --git a/docs/contribute/inference-family-surface.md b/docs/contribute/inference-family-surface.mdindex 3a0be052a..b61827798 100644--- a/docs/contribute/inference-family-surface.md+++ b/docs/contribute/inference-family-surface.md@@ -34,7 +34,7 @@ Most of the list is enforced by the type checker: the matches over the family en12. **Files, when the family reads them.** The job and the assignment carry them keyed by input name, as `JudgmentJob` and `JudgmentAssignment` carry `images` and `documents`, and the assignment's `referenced_uris()` returns each one, or a scoped run's read authorization never sees them. The worker's template method checks that the model reads them before its hook runs, raising the family's capability error (`JudgmentWorkerAbstract._check_can_read_files`), and runs the per-file format checks in `pipelex/cogt/inference/prompt_file_checks.py`, which the LLM family shares. What a model reads is the `inputs` of its spec in the backend's model file, so the capability is configuration, and an operator that knows its model at load time refuses a file input there too, as `PipeJudge` does.-13. **Silent: a default the gateway does not serve fails every boot.** The model manager checks, at boot, every handle a deck's presets and choice defaults name against the Pipelex Gateway's specs when the active routing profile sends that handle there. A family whose models only a bring-your-own-key backend serves must therefore ship **no** `choice_default` and **no** preset in the kit deck — an alias is fine, because aliases are not walked. Getting this wrong fails the boot of every installation, whether or not it ever uses the family.+13. **Silent: a default nothing serves boots cleanly and fails at run time.** A boot checks only that each alias a deck names exists, not that an enabled backend serves the model it resolves to, so a `default-*` alias pointing at a model no backend declares loads without complaint and fails the first pipe that relies on it with a `ModelNotFoundError`, for everyone running on their own keys. `tests/unit/pipelex/kit/test_shipped_deck_defaults.py` is the guard: every `default-*` alias of the shipped deck must reach, through aliases, presets and every entry of a waterfall, a model some backend file of the kit declares. Give a new family's defaults such a model, or ship no default, as `judgment` ships no `choice_default`.## A new backend for a familydiff --git a/docs/features/cli.md b/docs/features/cli.mdindex 6275e32ae..087289103 100644--- a/docs/features/cli.md+++ b/docs/features/cli.md@@ -1,6 +1,6 @@---title: "CLI"-description: "The Pipelex command-line interface for developing, validating, and running AI methods. Covers login, init, build, validate, run, graph, show, and more."+description: "The Pipelex command-line interface for developing, validating, and running AI methods. Covers init, build, validate, run, graph, show, and more."---# Command-Line Interface@@ -9,13 +9,12 @@ A comprehensive CLI for developing, validating, and running AI methods.## Overview-The `pipelex` CLI is the primary tool for working with Pipelex methods. It covers the full development lifecycle: authentication, initialization, building, validation, execution, inspection, and diagnostics.+The `pipelex` CLI is the primary tool for working with Pipelex methods. It covers the full development lifecycle: initialization, building, validation, execution, inspection, and diagnostics.## Core Commands| Command | Description ||---------|-------------|-| **`pipelex login`** | Authenticate with Pipelex Gateway via the browser and save your API key || **`pipelex init`** | Initialize configuration, backends, credentials, routing, and telemetry || **`pipelex update`** | Refresh the model deck and `backends/internal.toml` to match the installed pipelex version || **`pipelex doctor`** | Check configuration health and suggest fixes |diff --git a/docs/features/image-generation.md b/docs/features/image-generation.mdindex db499aaee..a4afe5992 100644--- a/docs/features/image-generation.md+++ b/docs/features/image-generation.md@@ -13,12 +13,10 @@ PipeImgGen generates images from text prompts using state-of-the-art models. Gen## Supported Models-Via **Pipelex Gateway**:+- **GPT-Image-2 / GPT-Image-1.5 / GPT-Image-1 / GPT-Image-1-mini** — OpenAI image generation models, served by the `openai` and `azure_openai` backends+- **Nano Banana / Nano Banana Pro / Nano Banana 2 / Nano Banana 2 Lite** — Google Gemini-based image generation, served by the `google` backend-- **GPT-Image-2 / GPT-Image-1.5 / GPT-Image-1 / GPT-Image-1-mini** — OpenAI image generation models-- **Nano Banana / Nano Banana Pro / Nano Banana 2 / Nano Banana 2 Lite** — Google Gemini-based image generation--Via **direct provider SDKs**:+The backends that generate images, each with your own key:- **OpenAI** — Direct OpenAI API for GPT Image models- **Google Gemini** — Native Google image generationdiff --git a/docs/features/index.md b/docs/features/index.mdindex afcd415ab..09bc984d8 100644--- a/docs/features/index.md+++ b/docs/features/index.md@@ -73,7 +73,7 @@ This section covers all Pipelex capabilities, organized by area.- **[CLI](cli.md)**- Full command-line interface: `login`, `init`, `doctor`, `build`, `validate`, `run`, `graph`, `show`, `which`, and more. Dry run mode, graph generation, and agent CLI.+ Full command-line interface: `init`, `doctor`, `build`, `validate`, `run`, `graph`, `show`, `which`, and more. Dry run mode, graph generation, and agent CLI.- **[plxt Formatter & Linter](plxt.md)**@@ -101,7 +101,7 @@ This section covers all Pipelex capabilities, organized by area.- **[Telemetry & Observability](telemetry.md)**- Production monitoring with Langfuse, OpenTelemetry, and PostHog integration. Gateway telemetry, span tracing, and custom destinations.+ Production monitoring with Langfuse, OpenTelemetry and PostHog integration: span tracing and custom destinations, all of them yours and off until you turn them on.- **[Cloud Storage](cloud-storage.md)**diff --git a/docs/features/pipe-operators.md b/docs/features/pipe-operators.mdindex 42379ed84..922f36748 100644--- a/docs/features/pipe-operators.md+++ b/docs/features/pipe-operators.md@@ -37,7 +37,7 @@ See [PipeImgGen reference](../building-methods/pipes/pipe-operators/PipeImgGen.m## PipeSearch-Web search with structured results and source citations. Powered by Linkup or Pipelex Gateway, results come back as typed SearchResult concepts ready for downstream processing.+Web search with structured results and source citations. Powered by Linkup, results come back as typed SearchResult concepts ready for downstream processing.See [PipeSearch reference](../building-methods/pipes/pipe-operators/PipeSearch.md).diff --git a/docs/features/telemetry.md b/docs/features/telemetry.mdindex d6a26d6c8..d809fd92f 100644--- a/docs/features/telemetry.md+++ b/docs/features/telemetry.md@@ -9,7 +9,7 @@ Production-ready monitoring for your AI methods.## Overview-Pipelex provides comprehensive telemetry and observability capabilities to monitor your AI methods in production. Track costs, latency, errors, and execution patterns across multiple destinations. Custom telemetry is opt-in and configurable per integration mode (CLI, pytest, API). When Pipelex Gateway is enabled as an inference backend, privacy-respecting Gateway metrics are collected automatically.+Pipelex provides comprehensive telemetry and observability capabilities to monitor your AI methods in production. Track costs, latency, errors, and execution patterns across multiple destinations. Every destination is yours, opt-in and configurable per integration mode (CLI, pytest, API), and nothing reports to Pipelex.## Langfuse Integration@@ -25,16 +25,12 @@ Event tracking and AI span tracing with fine-grained privacy controls. Choose be## Per-run attribution-A span or an event produced during a run is attributed to that run's own caller, not to the process that served it — which is what lets one deployment run many people's methods and still see each person's generations on their own timeline. The identity comes from the run: `user_id` is who started it, and the optional `extras` mapping carries your own opaque labels for it, such as which of your entities it belongs to, forwarded to PostHog as groups. Both are supplied where the run is, they ride the job through every nested pipe and across a process boundary, and they reach PostHog in the fields it reserves for identity — never as an event property or a span name. Work a host does for a known caller without it being a run is attributed the same way: validating a bundle, whose `pipe_dry_run` event and dry runs carry the caller the host passed, and an unhandled exception raised inside a run, which carries that run's caller out to the exception hook. Anything that names no caller of its own reports under the `user_id` you configured: an event outside any run and any caller's work, and a run on your own machine, whose caller is the literal `local` rather than a person a backend could tell apart — though its extras still ride as groups, because which entities a run belongs to is a separate question from who started it. Pipelex's own Gateway stream is attributed the same way: the run's `user_id` as it is, and its extras as groups. See [Telemetry Setup](../setup/telemetry.md) for how to supply the extras.--## Gateway Telemetry--When using Pipelex Gateway as your inference backend, privacy-respecting metrics are automatically collected: models used, token counts, latency, and error rates. This data is tied to your Gateway API key (hashed for security) and requires no additional configuration. Test runs are not collected: when the runtime boots in the `pytest` or `ci` integration mode, or in a session that loads Pipelex's shared pytest plugin, the Gateway stream stays off.+A span or an event produced during a run is attributed to that run's own caller, not to the process that served it — which is what lets one deployment run many people's methods and still see each person's generations on their own timeline. The identity comes from the run: `user_id` is who started it, and the optional `extras` mapping carries your own opaque labels for it, such as which of your entities it belongs to, forwarded to PostHog as groups. Both are supplied where the run is, they ride the job through every nested pipe and across a process boundary, and they reach PostHog in the fields it reserves for identity — never as an event property or a span name. Work a host does for a known caller without it being a run is attributed the same way: validating a bundle, whose `pipe_dry_run` event and dry runs carry the caller the host passed, and an unhandled exception raised inside a run, which carries that run's caller out to the exception hook. Anything that names no caller of its own reports under the `user_id` you configured: an event outside any run and any caller's work, and a run on your own machine, whose caller is the literal `local` rather than a person a backend could tell apart — though its extras still ride as groups, because which entities a run belongs to is a separate question from who started it. See [Telemetry Setup](../setup/telemetry.md) for how to supply the extras.## Privacy Controls- **DO_NOT_TRACK** — Universal telemetry disable flag, respected by all integrations-- **Configurable destinations** — Choose independently what data goes to PostHog, Langfuse, OTLP, or Gateway+- **Configurable destinations** — Choose independently what data goes to PostHog, Langfuse or OTLP- **Content redaction** — Control whether prompt/completion content is included in spans- **Mode-based gating** — Enable custom telemetry for CLI usage but disable it during unit testsdiff --git a/docs/features/web-search.md b/docs/features/web-search.mdindex da2ae3013..3c34d6120 100644--- a/docs/features/web-search.md+++ b/docs/features/web-search.md@@ -9,7 +9,7 @@ Structured web search integrated into your pipelines.## Overview-PipeSearch brings web search directly into your methods. Results come back as structured data with source citations, making them ready for downstream processing by LLMs or other pipes. Search is powered by Linkup (direct SDK) or via Pipelex Gateway.+PipeSearch brings web search directly into your methods. Results come back as structured data with source citations, making them ready for downstream processing by LLMs or other pipes. Search is powered by Linkup, through its SDK with your own `LINKUP_API_KEY`.## Key Capabilitiesdiff --git a/docs/get-started/configure-ai-providers.md b/docs/get-started/configure-ai-providers.mdindex 0a71d112d..bc1dbd7e5 100644--- a/docs/get-started/configure-ai-providers.md+++ b/docs/get-started/configure-ai-providers.md@@ -1,56 +1,24 @@----description: "Set up API keys for OpenAI, Anthropic, Mistral, and other providers to power your Pipelex executable AI methods — or use the gateway for instant access."+description: "Set up API keys for OpenAI, Anthropic, Mistral, OpenRouter and other providers, or a local model, to run your Pipelex executable AI methods on your own machine."---# Configure AI Providers## Configure API Access-To run pipelines with LLMs, you need to configure API access. **You have three options** - choose what works best for you:+To run pipelines on your own machine, Pipelex needs access to AI models. **You have two options**: your own provider keys, or models running locally. Nothing reports to Pipelex either way. If you would rather hold no provider keys at all, the [Quick Start](./quick-start.md) runs methods on the hosted Pipelex API instead.-### Option 1: Pipelex Gateway — Easiest and most Powerful for Getting Started--Get **free credits** for testing and development with a single API key for LLMs, document extraction, and image generation across all major providers (OpenAI, Anthropic, Google, Azure, open-source, and more). New models added constantly.--**Benefits:**--- No credit card required-- Access to OpenAI, Anthropic Claude, Google Gemini, xAI Grok, and more-- New models added constantly-- Perfect for development and testing-- Single API key for all models--**Setup:**--1. Get your API key at [app.pipelex.com](https://app.pipelex.com/)--2. Create a `.env` file in your project root:-- ```env- PIPELEX_GATEWAY_API_KEY=your-key-here- ```--3. Run `pipelex init` and accept the Gateway terms of service when prompted.--That's it! Your pipelines can now access any supported LLM. See [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) for the backends and models available.--!!! info "Terms of Service & Telemetry"- When using Pipelex Gateway, you'll be prompted to accept our terms of service. By using the Gateway, identified telemetry is automatically enabled (tied to your hashed API key) to help us monitor service quality and enforce fair usage.-- **We collect only technical data** (model names, token counts, latency, error rates). We do **NOT** collect your prompts, completions, or business data. See [Telemetry](../setup/telemetry.md) for details and trade-offs, and our [Privacy Policy](https://go.pipelex.com/privacy-policy) for more.--### Option 2: Bring Your Own API Keys+### Option 1: Bring Your Own API KeysUse your existing API keys from LLM providers. This is ideal if you:- Already have API keys from providers- Need to use specific accounts for billing- Have negotiated rates or enterprise agreements-- Prefer not to send any telemetry to Pipelex servers**Setup:**-Create a `.env` file in your project root with your provider keys:+Add your provider keys to a `.env` file in your project root, or to `~/.pipelex/.env` to use them in every project:```bash# OpenAI@@ -65,6 +33,9 @@ GOOGLE_API_KEY=...# MistralMISTRAL_API_KEY=...+# OpenRouter — one key for many providers' models+OPENROUTER_API_KEY=...+# FAL (for image generation)FAL_API_KEY=...@@ -82,7 +53,7 @@ AWS_SECRET_ACCESS_KEY=...AWS_REGION=...```-You only need to add keys for the providers you plan to use.+You only need to add keys for the providers you plan to use. A single OpenRouter key reaches models from many providers, but the deck's shipped defaults are OpenAI models that OpenRouter does not serve under those names: before running on OpenRouter alone, point the deck at models it serves, as [What the deck resolves to out of the box](#what-the-deck-resolves-to-out-of-the-box) describes.**Enable Your Providers:**@@ -108,7 +79,7 @@ When using your own keys, enable the corresponding backends:See [Inference Backend Configuration](../configuration/config-technical/inference-backend-config.md) for all options.-### Option 3: Local AI (No API Keys Required)+### Option 2: Local AI (No API Keys Required)Run AI models locally without any API keys. This is perfect if you:@@ -171,13 +142,14 @@ Learn more in our [Inference Backend Configuration](../configuration/config-tech### What the deck resolves to out of the box-The deck files `pipelex init` installs resolve the language, image-generation and document-extraction defaults to models the Pipelex Gateway serves from Azure, so a fresh install runs inside one provider's scope without you choosing anything:+The deck files `pipelex init` installs resolve their defaults to these models, and the default routing profile, `all_enabled_backends`, sends each model to the first enabled backend that serves it:-- **Language models** — the whole ladder is the GPT-5.6 range: the premium tier and `best-gpt` are GPT-5.6 Sol, the general and large-context tiers are GPT-5.6 Terra, and the small tiers are GPT-5.6 Luna.-- **Image generation** — the general and premium tiers are GPT Image 2, the small tier is GPT Image 1 mini.-- **Document extraction** — Azure Document Intelligence. `default-text-from-pdf` and `default-no-inference` are the exception within that family: they read the PDF locally with pypdfium2 and call no model, so they need no key of any kind.+- **Language models** — the whole ladder is the GPT-5.6 range, served by `openai` and `azure_openai`: the premium tier and `best-gpt` are GPT-5.6 Sol, the general and large-context tiers are GPT-5.6 Terra, and the small tiers are GPT-5.6 Luna.+- **Image generation** — the general and premium tiers are GPT Image 2, the small tier is GPT Image 1 mini, both served by `openai` and `azure_openai`.+- **Document extraction** — `default-extract-document` tries Mistral OCR, which needs your Mistral key, and otherwise reads a PDF's text layer locally with pypdfium2, which needs no key but recovers no text from a scanned page. `default-extract-image` and `default-premium` are Mistral OCR only. `default-text-from-pdf` and `default-no-inference` always read the PDF locally and call no model. See [Document Extraction](../features/document-extraction.md#the-default-extractor).+- **Web search and web pages** — everything in `4_search_deck.toml`, and `default-extract-web-page` in `3_extract_deck.toml`, resolve to Linkup, so a method that searches the web or extracts from a web page needs a Linkup key.-**Where the deck leaves Azure, it goes to Linkup**, and it does so in two families rather than one. Azure serves no search model, so everything in `4_search_deck.toml` resolves to Linkup; and `default-extract-web-page` in `3_extract_deck.toml` resolves to `linkup-fetch`, which pulls a web page through Linkup rather than through Azure Document Intelligence. A method that searches the web, or that extracts from a web page, needs a Linkup key or the Gateway.+With an OpenAI or Azure OpenAI key alone, the language and image defaults run as shipped. With any other provider, OpenRouter included, point the aliases at models it serves, as the next paragraph describes; the models each provider serves are listed under `~/.pipelex/inference/backends/`. A few presets name a model directly rather than through an alias, `retrieval-cheap`, `retrieval-premium`, `engineering-code-cheap` and `engineering-code-cheaper`, so they need overriding too. On OpenRouter, the presets that set a reasoning effort, `deep-analysis`, `quick-reasoning` and `retrieval-premium`, also need overriding without it, because its models are declared without reasoning support.Nothing about this locks you in. The deck is a vocabulary of aliases and presets, not a provider commitment: point any of them at a model from any backend you have enabled, by editing `x_custom_llm_deck.toml`, which `pipelex update` never touches. That is also how you bring back an alias the shipped deck does not define.diff --git a/docs/get-started/mthds-language-tutorial.md b/docs/get-started/mthds-language-tutorial.mdindex ce64f5865..05b71a57e 100644--- a/docs/get-started/mthds-language-tutorial.md+++ b/docs/get-started/mthds-language-tutorial.md@@ -7,17 +7,16 @@ description: "Learn the declarative MTHDS language step by step — pipes, sequeThis tutorial walks you through writing `.mthds` files manually, step by step.-Set up your project and get free AI access:+Set up your project:```bashuv tool install "pipelex[cli]"pipelex init-pipelex login```The `cli` extra installs Rich, which the `pipelex` command renders its output through.-`pipelex init` creates your project configuration. `pipelex login` opens your browser to authenticate with Pipelex Gateway (free). See [Configure AI Providers](./configure-ai-providers.md) for other options: bring your own keys, local AI, etc.+`pipelex init` creates your project configuration. To run your methods, add your own provider keys to `~/.pipelex/.env`, or point Pipelex at a local model: see [Configure AI Providers](./configure-ai-providers.md).!!! tip "VS Code Extension"We **highly** recommend installing the Pipelex extension for `.mthds` syntax highlighting and flowchart visualization:diff --git a/docs/index.md b/docs/index.mdindex 9867fcc7c..8893b940a 100644--- a/docs/index.md+++ b/docs/index.md@@ -99,7 +99,7 @@ From here, Pipelex handles model routing across 60+ models, structured output pa- :material-cloud-check: **[60+ AI Models](./features/llm-integration.md)**- One gateway key or bring-your-own: OpenAI, Anthropic, Mistral, Google, Deepseek, Hugging Face, and more.+ Your own provider keys for OpenAI, Anthropic, Mistral, Google, Deepseek, Hugging Face and more, or local models.- :material-check-decagram: **[Validation and Dry Run](./features/validation-dry-run.md)**diff --git a/docs/reliability/automatic-retries.md b/docs/reliability/automatic-retries.mdindex ab3934d2d..9f6109b2a 100644--- a/docs/reliability/automatic-retries.md+++ b/docs/reliability/automatic-retries.md@@ -20,7 +20,7 @@ transport_max_retries = 2`transport_max_retries` (default `2`) is the number of retries attempted **on top of** the initial request. A value of `2` allows up to 3 attempts total. What Pipelex sets uniformly is this *retry count*; the exact set of failures a retry fires on is the provider SDK's own transient-failure set — broadly connection errors, request timeouts (`408`), rate limits (`429`), and server errors (`5xx`), with minor per-SDK variation (Google's client, for instance, omits `409`). Pipelex's own SDK-less path (the raw-HTTP Azure image-generation client) pins this set explicitly to `408` / `409` / `429` / `5xx`. A `Retry-After` response header is honored where the SDK or that path supports it.-This setting is wired into every inference SDK client that exposes a client-side retry budget — Anthropic, OpenAI / Azure OpenAI, the Pipelex Gateway LLM clients, Mistral, and Google — as well as the raw-HTTP Azure image-generation path. So the retry posture is one deliberate policy across those provider SDK clients. (The Pipelex Gateway's document-extraction and image-generation calls go through the Portkey SDK, which has no client-side retry budget; transport retries for those are owned by the gateway itself.)+This setting is wired into every inference SDK client that exposes a client-side retry budget — Anthropic, OpenAI / Azure OpenAI and the other backends reached through the OpenAI SDK, Portkey, Mistral, Google, and TypeSafe — as well as the raw-HTTP Azure image-generation path. So the retry posture is one deliberate policy across those provider SDK clients.!!! info "Transport retry is not pipeline retry"Tier 1 retries a *single* HTTP request to a provider. It does not re-run a pipe, re-run a step, or restart a pipeline. If a call still fails after its transport retries are exhausted, the error surfaces.diff --git a/docs/setup/telemetry.md b/docs/setup/telemetry.mdindex 0eaa6d030..51eeaba4d 100644--- a/docs/setup/telemetry.md+++ b/docs/setup/telemetry.md@@ -1,57 +1,24 @@----description: "Understand Pipelex telemetry options — what data is collected, how to opt out, and how privacy works for gateway and anonymous usage streams."+description: "Understand Pipelex telemetry — nothing reports to Pipelex, and the streams you configure yourself are off until you turn them on."---# Telemetry-Pipelex supports two independent telemetry streams that serve different purposes. Understanding how they work helps you make informed decisions about data collection.+**Pipelex sends no telemetry to Pipelex.** The only telemetry the runtime produces is yours: you configure it in `telemetry.toml`, it is off until you turn it on, and it goes to analytics and monitoring systems you operate.-## Two Telemetry Streams+## Your Telemetry Streams-### 1. Gateway Telemetry (Pipelex-Controlled)--When you use **Pipelex Gateway** as your inference backend, identified telemetry is **automatically enabled**. This telemetry is tied to your Gateway API key (hashed for security) and operates independently from your `telemetry.toml` settings.--Test runs are the exception, and either of two signals marks one. The first is the `pytest` or `ci` integration mode. The second is a test run mode, which Pipelex's shared pytest plugin (`pipelex.test_extras.shared_pytest_plugins`) sets for every session that loads it, when pytest configures and before it collects anything, so a suite booting with `Pipelex.make()` in the default mode is covered too, whether it boots in a fixture, at test-module import or in a pytest hook that runs after `pytest_configure`. What it cannot reach is anything pytest runs before it configures: the body of every `conftest.py` loaded at startup, which means the ones along the paths given on the command line or in `testpaths` and not only the one that loads the plugin, and the hooks that run before `pytest_configure`, such as `pytest_addoption` or `pytest_cmdline_main`. Boot in a fixture there. In a test run the Gateway stream stays off, so a project's test suite sends nothing to Pipelex even with the Gateway enabled: the boot does not look for `PIPELEX_GATEWAY_API_KEY` on the stream's behalf, and `DO_NOT_TRACK` does not conflict with the Gateway. The Gateway's Portkey request logging and trace correlation follow the stream, so the `debug` setting on the `pipelex_gateway` backend and the `[pipelex_gateway.portkey]` force flags in `telemetry.toml` have no effect in a test run either, even where your own custom stream is allowed in that mode.--A run that names a caller of its own is attributed to that caller on this stream too: the run's `user_id` is sent as the `distinct_id` exactly as you supplied it, and its `extras` ride the capture as groups. Anything that names no caller reports under your key's hash.--**What we collect:**--- Model names used (e.g., `gpt-5.4`, `claude-4.5-sonnet`) and parameters-- Pipe types (e.g., `PipeLLM`, `PipeSequence`, etc.)-- Token counts (input/output)-- Latency metrics-- Error rates (without error details)--**What we do NOT collect:**--- Your prompts or completions-- Your pipe codes or output class names-- File contents or business data--This telemetry allows us to:--- Monitor and improve service quality-- Enforce fair usage limits and prevent abuse-- Provide you with usage insights and better support--!!! info "Gateway Telemetry is Optional"- Using Pipelex Gateway is entirely optional. If you prefer not to send telemetry to Pipelex servers, simply use your own API keys with direct provider backends (OpenAI, Anthropic, Azure, Bedrock, etc.).--### 2. Custom Telemetry (User-Controlled)--Custom telemetry is configured in `.pipelex/telemetry.toml` and allows you to send observability data to **your own** analytics and monitoring systems:+`telemetry.toml` sends observability data to **your own** analytics and monitoring systems:- **PostHog**: Event tracking and AI span tracing with privacy controls- **Langfuse**: Full LLM observability (receives full span data)- **OTLP**: Send spans to any OpenTelemetry-compatible backend (receives full span data)-Custom telemetry is completely independent from Gateway telemetry—you can use both, either, or neither.+Each integration mode allows these streams or not: by default the `cli`, `docker`, `fastapi`, `mcp` and `n8n` modes allow them, and `pytest`, `ci` and `python` do not, so a test suite sends nothing unless you allow its mode under `[telemetry_allowed_modes]` in `telemetry.toml`. [Telemetry Configuration](../configuration/config-practical/telemetry-config.md) describes each destination's settings.### Pipelex's spans in your process-Either stream makes Pipelex trace a run: the Gateway stream whenever it is on, and yours when AI span tracing is enabled on your PostHog. Pipelex's tracer stays its own and never becomes the global one, so your own spans never reach Pipelex's exporters, and **Pipelex never makes its spans current in your process's OpenTelemetry context**:+Pipelex traces a run when your PostHog stream is on (`mode` is `anonymous` or `identified`) and its AI span tracing is enabled (`[custom_posthog.tracing]`); the same tracer feeds your Langfuse and OTLP destinations. Pipelex's tracer stays its own and never becomes the global one, so your own spans never reach Pipelex's exporters, and **Pipelex never makes its spans current in your process's OpenTelemetry context**:- Your own instrumentation, an HTTP client's or a provider SDK's, keeps opening its spans under your own current span, in your own trace, while a Pipelex run goes. It is never re-parented under a pipe or an LLM call, nor kept by a sampler that follows its parent because a Pipelex span was sampled.- An error tracker that reads OpenTelemetry's current span sees yours, not Pipelex's.@@ -107,13 +74,7 @@ The `DO_NOT_TRACK` environment variable provides a universal way to disable **alexport DO_NOT_TRACK=1```-When set, this disables:--- Gateway telemetry (but note: outside a test run, Gateway won't work without telemetry)-- All custom telemetry destinations--!!! warning "Gateway Requires Telemetry"- If you set `DO_NOT_TRACK=1` while using Pipelex Gateway, the Gateway will not function, except in a test run, where the Gateway stream is off anyway. Use direct provider backends instead if you need to disable all telemetry.+When set, it disables every telemetry destination, whatever `telemetry.toml` says, and the request logging and tracing of your own Portkey backend too, `[custom_portkey]`'s force flags included.## Attaching your own labels to a run@@ -125,7 +86,7 @@ The run-level metadata every run carries (`RunMetadata`) has an `extras` field:extras = {"organization": "org_acme"}```-On **your own** PostHog stream, every span and every event the run produces is then captured under that run's `user_id`, with its extras attached through PostHog's own groups facet — so a generation made for one of your customers appears on that customer's timeline and inside their organization, instead of under one identity per deployment. The same values reach every OpenTelemetry exporter as the span attributes `pipelex.run.user_id` and `pipelex.run.extras`, and Langfuse receives the user id in the field it reserves for it. Pipelex's own Gateway stream receives the same `user_id` and groups, as described above.+On **your own** PostHog stream, every span and every event the run produces is then captured under that run's `user_id`, with its extras attached through PostHog's own groups facet — so a generation made for one of your customers appears on that customer's timeline and inside their organization, instead of under one identity per deployment. The same values reach every OpenTelemetry exporter as the span attributes `pipelex.run.user_id` and `pipelex.run.extras`, and Langfuse receives the user id in the field it reserves for it.Anything that names no caller of its own keeps reporting under the `user_id` you configured in `telemetry.toml`, which is what that setting now means: the identity of everything that is not one caller's run. That covers an event outside any run — a CLI command listing your pipes — and a run whose caller is not a distinguishable person either: a run on your own machine is attributed to the literal `local`, the same string on every machine, so Pipelex declines it as an identity and uses your configured id instead. The same goes for `single-tenant`, which pipelex-api states for every run when it is deployed with no users. Per-run attribution is for a host that passes a real `user_id` per run.@@ -159,8 +120,8 @@ These are labels for grouping, not content: they land in the reserved identity fWe take your privacy seriously:-- **Gateway telemetry** never collects prompts, completions, or business data-- **Custom telemetry** gives you full control over what data is captured+- Nothing reports to Pipelex: every stream is one you configure, sent to a system you operate+- Your `telemetry.toml` gives you full control over what data is captured- PostHog tracing includes privacy controls to redact sensitive content- All telemetry can be completely disableddiff --git a/docs/tools/logging.md b/docs/tools/logging.mdindex 64baa1ad2..8bd973fe9 100644--- a/docs/tools/logging.md+++ b/docs/tools/logging.md@@ -137,7 +137,7 @@ The runtime holds each span it starts as the active Pipelex span from its start- **A pipe run's span**: `live_run_pipe` starts the span, then runs the pipe with it held, so a line logged by the pipe's own work, a controller's or an operator's and the lines of the extract, image-generation, search or function call an operator makes, names the pipe's span under `pipelex.*`, and a nested pipe's lines name the nested pipe's span until it returns. The line announcing the run is logged before the span starts, under the enclosing span.- **An LLM call's span**: the LLM worker runs the provider call with its generation span held, so the lines of the call and of its failure name that span under `pipelex.*`, a provider SDK's own lines included.-The runtime starts these spans only when it traces a live run, which it does once telemetry has created its tracer: with the Pipelex Gateway's telemetry on, which is automatic when the Gateway serves your inference, or with AI span tracing enabled on your own PostHog, as [Telemetry](../setup/telemetry.md#pipelexs-spans-in-your-process) describes. Otherwise it holds no span, as it does when its tracer is a no-op one, under `OTEL_SDK_DISABLED` for instance, which starts no span of its own, and a line carries no `pipelex.*` fields. A boot line held until the sink arrives is replayed in the context it was logged in, so it keeps both its spans, and a thread started with `asyncio.to_thread` or `contextvars.copy_context` inherits both from the code that started it.+The runtime starts these spans only when it traces a live run, which it does once telemetry has created its tracer, with AI span tracing enabled on your own PostHog, as [Telemetry](../setup/telemetry.md#pipelexs-spans-in-your-process) describes. Otherwise it holds no span, as it does when its tracer is a no-op one, under `OTEL_SDK_DISABLED` for instance, which starts no span of its own, and a line carries no `pipelex.*` fields. A boot line held until the sink arrives is replayed in the context it was logged in, so it keeps both its spans, and a thread started with `asyncio.to_thread` or `contextvars.copy_context` inherits both from the code that started it.Holding a span is in-process plumbing, like binding the log context: it lives in a context variable of Pipelex's own. **Pipelex never makes its spans current in your process's OpenTelemetry context**, so your own instrumentation, an HTTP client's or a provider SDK's, is never re-parented under a Pipelex span, and an error tracker that reads OpenTelemetry's current span sees yours, not Pipelex's. A span's children take their parent from the job metadata, so a step running in another process gets the parent it always did, and a line that process logs carries `pipelex.*` fields only while the process runs a span of its own, the LLM call's for instance.diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.mdindex 45b9f9dfe..e3bf3df8b 100644--- a/docs/under-the-hood/error-model.md+++ b/docs/under-the-hood/error-model.md@@ -424,7 +424,7 @@ The `validate` surface — both the bare `pipelex validate {bundle,method,pipe}`**The verdict lives in the structured `is_valid` field, not the exit code.** The exit code is a convenience signal for naive shell/CI/Makefile use (`set -e`, `cmd && next`, `if cmd; then`); machine consumers (hooks, the Codex hook, runners) MUST read `is_valid` (and `error_domain`) from the JSON for their block/warn decisions rather than branching on the exit code. Decoupling the verdict from the exit code is what keeps any future exit-code change non-breaking. The 1-vs-2 split is also additive for flat consumers: both stay non-zero, so anything that only tests zero-vs-non-zero is unaffected.-Implementation: the agent CLI threads `exit_code` through `agent_error(...)` (`agent_output.py`, default 1); the validate commands pass `exit_code=2` at every no-verdict site and keep the default 1 on the `ValidateBundleError` arm and the signature gate. The bare CLI sets the code directly via `typer.Exit(...)` in `cli/commands/validate/*` and via the `exit_code` parameter on `handle_model_availability_error` in `cli/error_handlers.py`. Every refusal of the bundle is a negative verdict on both CLIs: `validate bundle` and `validate method` validate through the shared cascade, and `validate pipe` and `validate --all` load their libraries through it too, so an unknown model or any other load-time refusal exits 1 with the invalid-bundle output rather than a traceback or the no-verdict exit 2. The bare `validate pipe` and `validate --all` also run their dry run through it, so a pipe whose dry run fails is rendered like any other invalid bundle; the agent CLI answers that failure with its `DryRunError` envelope, `is_valid: false` and exit 1. Shared boot handlers (`make_pipelex_for_cli`'s gateway/inference/telemetry/model-deck-preset paths) stay exit 1 — they are shared across `run`/`build`/`validate` and out of the validate-policy scope.+Implementation: the agent CLI threads `exit_code` through `agent_error(...)` (`agent_output.py`, default 1); the validate commands pass `exit_code=2` at every no-verdict site and keep the default 1 on the `ValidateBundleError` arm and the signature gate. The bare CLI sets the code directly via `typer.Exit(...)` in `cli/commands/validate/*` and via the `exit_code` parameter on `handle_model_availability_error` in `cli/error_handlers.py`. Every refusal of the bundle is a negative verdict on both CLIs: `validate bundle` and `validate method` validate through the shared cascade, and `validate pipe` and `validate --all` load their libraries through it too, so an unknown model or any other load-time refusal exits 1 with the invalid-bundle output rather than a traceback or the no-verdict exit 2. The bare `validate pipe` and `validate --all` also run their dry run through it, so a pipe whose dry run fails is rendered like any other invalid bundle; the agent CLI answers that failure with its `DryRunError` envelope, `is_valid: false` and exit 1. Shared boot handlers (`make_pipelex_for_cli`'s telemetry-config and model-deck-preset paths) stay exit 1 — they are shared across `run`/`build`/`validate` and out of the validate-policy scope.### APIdiff --git a/docs/under-the-hood/inference-backend-plugins.md b/docs/under-the-hood/inference-backend-plugins.mdindex e3edd7f26..4087c617d 100644--- a/docs/under-the-hood/inference-backend-plugins.md+++ b/docs/under-the-hood/inference-backend-plugins.md@@ -7,7 +7,7 @@ description: "How Pipelex discovers inference backends through the plugin seam,Every model call in Pipelex — an LLM completion, an image generation, a document extraction, a web search, a judgment — is served by an **inference worker**. Which worker handles a given model is decided entirely by data: a model's `sdk` field selects a backend, and a **backend plugin** is what teaches Pipelex how to build the worker for that `sdk`.-Core names no backend by import or by string. The built-in drivers (OpenAI, Gateway, Anthropic, Mistral, Bedrock, Google, FAL, HuggingFace, Docling, …) are plugins too — they ride the exact same seam an out-of-tree plugin would. This page documents that seam, the **Inference SPI** a plugin compiles against, and how to write one.+Core names no backend by import or by string. The built-in drivers (OpenAI, Anthropic, Mistral, Bedrock, Google, FAL, HuggingFace, OpenRouter, Portkey, Docling, …) are plugins too — they ride the exact same seam an out-of-tree plugin would. This page documents that seam, the **Inference SPI** a plugin compiles against, and how to write one.---@@ -65,7 +65,7 @@ registrar.add_inference_backend(A registry key is `(family, sdk)`. The same `sdk` string may appear in two families (e.g. `google` serves both `LLM` and `IMG_GEN`); they are distinct keys. A duplicate `(family, sdk)` fails loud with `DuplicateInferenceBackendError` naming **both** contributing plugins.-One plugin may register across several families from a single `register` — the built-in `gateway` plugin serves the four inference families, `mistral` serves `LLM` + `EXTRACT`, `linkup` serves `EXTRACT` + `SEARCH`, and `typesafe` serves `JUDGMENT` alone. This is the cross-family-vendor coordination point: one plugin, many backends. `DOC_GEN` is the family of the document engines a `PipeDocGen` step prints with, local libraries rather than inference, which the built-in `reportlab` plugin and the Pipelex document generation plugin serve (see [Document Engine Plugins](document-engine-plugins.md)).+One plugin may register across several families from a single `register` — `openai` and `google` serve `LLM` + `IMG_GEN`, `mistral` serves `LLM` + `EXTRACT`, `linkup` serves `EXTRACT` + `SEARCH`, and `typesafe` serves `JUDGMENT` alone. This is the cross-family-vendor coordination point: one plugin, many backends. `DOC_GEN` is the family of the document engines a `PipeDocGen` step prints with, local libraries rather than inference, which the built-in `reportlab` plugin and the Pipelex document generation plugin serve (see [Document Engine Plugins](document-engine-plugins.md)).A plugin whose backend runs without an external service, such as a document engine, may also declare the models it serves rather than leave them to a backend file. `add_internal_model(name=…, spec=…)` declares a model in the `internal` backend, the spec being the table a backend file would hold for it, and `add_doc_gen_default(doc_gen_format=…, source=…, model=…)` declares the model deck's default engine for a document format and source. Both are stored at registration and merged by the model manager at boot, and [Document Engine Plugins](document-engine-plugins.md#registering-an-engine) gives their rules.@@ -304,7 +304,7 @@ What an out-of-tree backend plugin imports *is* the contract. The published surf| `JobMetadata` | `pipelex.system.job_metadata` | the run and the step a job belongs to || `ReportingProtocol` | `pipelex.reporting.reporting_protocol` | where a worker reports its usage |-The SPI is a documented, versioned **module/symbol list** gated by `PLUGIN_API_VERSION` — not an `__init__.py` re-export shim (the repo bans re-exports; import by full path). Anything a plugin needs to import outside this surface is a design gap to resolve, not an accident to live with: either the symbol is published here, or the plugin keeps its own copy of it. The built-in provider plugin for the hosted gateway is held to it by a test that scans its imports against this table.+The SPI is a documented, versioned **module/symbol list** gated by `PLUGIN_API_VERSION` — not an `__init__.py` re-export shim (the repo bans re-exports; import by full path). Anything a plugin needs to import outside this surface is a design gap to resolve, not an accident to live with: either the symbol is published here, or the plugin keeps its own copy of it. Pipelex's own out-of-tree inference plugin is held to it by a test, in that plugin's repository, that scans its imports against this table.---diff --git a/docs/under-the-hood/init-cli-flows.md b/docs/under-the-hood/init-cli-flows.mdindex 85a6962ae..0cc8b5dd3 100644--- a/docs/under-the-hood/init-cli-flows.md+++ b/docs/under-the-hood/init-cli-flows.md@@ -33,7 +33,7 @@ These two categories are managed by separate steps. `init_config()` copies only| `pipelex init telemetry` | `telemetry` | Telemetry config template copy || `pipelex init credentials` | `credentials` | Credential setup for enabled backends |-All commands except `agreement` and `credentials` perform a **full reset** (overwrite existing files) — every setting in the file is replaced by the template's. That is why init is not the answer to a configuration file that has fallen behind the current schema: [`pipelex migrate`](../tools/cli/migrate.md) rewrites such a file in place and keeps what is in it.+All commands except `credentials` perform a **full reset** (overwrite existing files) — every setting in the file is replaced by the template's. That is why init is not the answer to a configuration file that has fallen behind the current schema: [`pipelex migrate`](../tools/cli/migrate.md) rewrites such a file in place and keeps what is in it.### Inputs@@ -67,9 +67,6 @@ All commands except `agreement` and `credentials` perform a **full reset** (overflowchart TDSTART([pipelex init]) --> FOCUS{focus?}- FOCUS -- agreement --> AGREE[_init_agreement]- AGREE --> DONE([Done])-FOCUS -- credentials --> CREDS_DIRECT["prompt_credentials<br/>Prompt for missing API keys"]CREDS_DIRECT --> DONE@@ -258,7 +255,7 @@ Copies a telemetry template and prints instructions. No interactive prompts. Whi| Constant | Contents | Reason ||----------|----------|--------|-| `INIT_SKIP_FILES` | Every name in `GIT_IGNORED_CONFIG_FILES` (`pipelex/kit/paths.py` is the list: the personal overrides, the service file, the generated gateway model pages, the custom deck files) plus `telemetry.toml` and `.DS_Store` | Git-ignored, auto-generated, or managed by other steps |+| `INIT_SKIP_FILES` | Every name in `GIT_IGNORED_CONFIG_FILES` (`pipelex/kit/paths.py` is the list: the personal overrides, `telemetry.project.toml`, the retired `pipelex_service.toml` an earlier release wrote, the custom deck files) plus `telemetry.toml` and `.DS_Store` | Git-ignored, auto-generated, or managed by other steps || `INIT_SKIP_DIRS` | `inference` | Managed independently by inference step |### Source Modulesdiff --git a/docs/under-the-hood/reasoning-controls.md b/docs/under-the-hood/reasoning-controls.mdindex 2248a2ba3..589bb5891 100644--- a/docs/under-the-hood/reasoning-controls.md+++ b/docs/under-the-hood/reasoning-controls.md@@ -282,7 +282,7 @@ If the level map returns `"disabled"` (e.g., for `NONE` effort), thinking is disTemperature is passed normally to the Google API regardless of reasoning mode.!!! note "VertexAI Backend"- Reasoning controls are not implemented for Gemini models on the VertexAI backend because Google favors the newer Gen-AI SDK. The VertexAI backend uses `sdk = "openai"` (the OpenAI-compatible endpoint), which routes through the OpenAI worker and does not expose Google's native thinking controls (`thinking_budget` / `thinking_level`). For Gemini reasoning support, use the `google` backend with the native Gen-AI SDK. Or better yet, use the Pipelex Gateway.+ Reasoning controls are not implemented for Gemini models on the VertexAI backend because Google favors the newer Gen-AI SDK. The VertexAI backend uses `sdk = "openai"` (the OpenAI-compatible endpoint), which routes through the OpenAI worker and does not expose Google's native thinking controls (`thinking_budget` / `thinking_level`). For Gemini reasoning support, use the `google` backend with the native Gen-AI SDK.### Mistral@@ -317,11 +317,12 @@ Bedrock native models using the `bedrock_aioboto` SDK do not support reasoning p### Gateway and Proxy Backends-Gateway and proxy backends (Azure OpenAI, Portkey, BlackBoxAI, Pipelex Gateway) route API calls through an intermediary but use the same provider worker classes as direct backends. Their reasoning capabilities depend on the `sdk` field in each model's backend TOML, which determines which worker handles the request.+Gateway and proxy backends (Azure OpenAI, Portkey, BlackBoxAI, OpenRouter) route API calls through an intermediary but use the same provider worker classes as direct backends. Their reasoning capabilities depend on the `sdk` field in each model's backend TOML, which determines which worker handles the request.- **Azure OpenAI** uses `sdk = "azure_openai_responses"`, routing through the OpenAI Responses worker. Reasoning models declare `thinking_mode = "manual"` and use OpenAI-style `reasoning_effort`.- **Portkey** uses `portkey_completions` or `portkey_responses` SDKs, both routing through OpenAI workers. All models — including Anthropic and Google models proxied via Portkey — follow OpenAI reasoning semantics.- **BlackBoxAI** uses `sdk = "openai"` or `"openai_responses"`. Proxied models follow OpenAI reasoning semantics.+- **OpenRouter** uses `sdk = "openai"` for its language models, which declare `thinking_mode = "none"`, so the kit's OpenRouter models take no reasoning controls.!!! noteWhen a provider's models are accessed through a gateway using an OpenAI-compatible SDK, the reasoning controls follow OpenAI semantics (`reasoning_effort`) rather than the provider's native semantics. For native reasoning controls (e.g., Anthropic thinking budgets, Google thinking levels), use the direct provider backend.diff --git a/docs/under-the-hood/secrets-provider-plugins.md b/docs/under-the-hood/secrets-provider-plugins.mdindex df22dd3ed..4177c959b 100644--- a/docs/under-the-hood/secrets-provider-plugins.md+++ b/docs/under-the-hood/secrets-provider-plugins.md@@ -43,7 +43,7 @@ There is no `match runtime.secrets.method:` anywhere in boot — the token set i### Where in boot secrets is resolved-Secrets is the **first capability resolved out of the registrar** — right after plugin discovery, after the gateway/terms precondition gate, before the log sink and well before storage selection. Three consequences that this ordering guarantees:+Secrets is the **first capability resolved out of the registrar** — right after plugin discovery and the unknown-orchestrator check, before the log sink and well before storage selection. Three consequences that this ordering guarantees:- **The log sink**, its first consumer, receives the provider as the `secrets_provider` keyword of its factory, so a sink's settings can name secrets: the built-in `otlp` sink's header values and the `gcp` sink's key path accept `${…}` placeholders (see [Log Sink Plugins](log-sink-plugins.md#logsinkfactoryfn)). The provider goes on the hub only after the sink is installed, so the keyword is the one way a sink reaches it. Because the sink does not exist yet, what the provider logs while it is built is held and replayed into the sink, and a provider whose factory raises stops the boot with those held lines written to stderr, redacted, rather than through the configured sink.- **Telemetry** receives the config-selected secrets provider when it is constructed.diff --git a/pipelex/cogt/model_backends/model_spec_factory.py b/pipelex/cogt/model_backends/model_spec_factory.pyindex 24e5ece5a..9c33b6520 100644--- a/pipelex/cogt/model_backends/model_spec_factory.py+++ b/pipelex/cogt/model_backends/model_spec_factory.py@@ -31,7 +31,7 @@ class InferenceModelSpecBlueprint(ConfigModel):listed_constraints: list[ListedConstraint] = Field(default_factory=empty_list_factory_of(ListedConstraint))valued_constraints: dict[ValuedConstraint, Any] = Field(default_factory=empty_dict_factory_of(ValuedConstraint))rules: ImgGenModelRules | None = None- # Provider-side route for models the worker calls by raw path (today: the gateway image models). Our own+ # Provider-side route for models a worker calls by raw path (no kit backend declares one; an out-of-tree plugin may). Our own# routing metadata, not a request header — which is why it is a declared field and not an `extra_headers` entry.endpoint_path: str | None = Nonediff --git a/pipelex/cogt/models/model_deck.py b/pipelex/cogt/models/model_deck.pyindex ba3b0f0ea..9ac4adeb4 100644--- a/pipelex/cogt/models/model_deck.py+++ b/pipelex/cogt/models/model_deck.py@@ -95,11 +95,11 @@ class JudgmentDeckBlueprint(ConfigModel):"""The judgment family's half of the deck.``choice_default`` is optional here where every other family requires one, and the reason is- that no judgment model is served by default. The other families' defaults name a handle the- Pipelex Gateway carries, so they always resolve; a judgment backend is brought by the user, and- a deck naming a default that nothing serves fails the gateway's own membership check at boot —- for every user, whether or not they ever ask for a judgment. Absent says the honest thing: no- model is the default, so a judgment names its own until one is served.+ that no judgment model is served by default: the judgment backend is one the user brings. A+ default that nothing serves boots cleanly and fails the first judgment that relies on it, and+ ``tests/unit/pipelex/kit/test_shipped_deck_defaults.py`` refuses a shipped default no kit+ backend declares. Absent says the honest thing: no model is the default, so a judgment names+ its own until one is served."""aliases: dict[str, str] = Field(default_factory=dict)diff --git a/pipelex/system/telemetry/posthog_span_exporter.py b/pipelex/system/telemetry/posthog_span_exporter.pyindex 36ed6aea4..6c6574ae0 100644--- a/pipelex/system/telemetry/posthog_span_exporter.py+++ b/pipelex/system/telemetry/posthog_span_exporter.py@@ -30,10 +30,9 @@ class PostHogSpanExporter(SpanExporter):Applies redaction rules from TelemetryRedactionConfig before sending events,and attributes each span to the run that produced it — see- :mod:`pipelex.system.telemetry.telemetry_identity`. One class serves both- streams: the operator's exporter is built with their configured id and the- policy their mode implies, Pipelex's with the gateway-key hash as its- fallback and the direct policy.+ :mod:`pipelex.system.telemetry.telemetry_identity`. The exporter is built+ with the operator's configured id as its fallback and the policy their mode+ implies."""def __init__(@@ -53,7 +52,7 @@ class PostHogSpanExporter(SpanExporter):run_identity_policy: What this stream may do with a span's own user.`NONE` on an operator's stream in `anonymous` mode, where theoperator chose not to identify people at all; `DIRECT`- otherwise, Pipelex's stream included.+ otherwise.redaction_config: What this stream is allowed to see."""self.posthog_client = posthog_clientdiff --git a/pipelex/system/telemetry/telemetry_identity.py b/pipelex/system/telemetry/telemetry_identity.pyindex 3ede5c02e..1f7b11381 100644--- a/pipelex/system/telemetry/telemetry_identity.py+++ b/pipelex/system/telemetry/telemetry_identity.py@@ -2,7 +2,7 @@**Identity is a fact of the run, not of the process.** A `TelemetryManager` bindsone `distinct_id` when it is built — the operator's configured `user_id` on their-own stream, the gateway-key hash on Pipelex's — and before this module every span+own stream — and before this module every spanand every event carried that one id. On a host that serves many callers from oneprocess that is exactly wrong: every tenant's generations land under one identityper deployment. The runtime already knows who each run belongs to, on@@ -75,7 +75,7 @@ if TYPE_CHECKING:## So for telemetry these mean "this run names nobody", and the stream's own# fallback answers instead: the operator's configured `user_id` on their stream,-# the gateway-key hash on Pipelex's — which is exactly what a CLI run reported+# which is exactly what a CLI run reported# under before per-run attribution existed, and still does._NON_DISTINGUISHING_RUN_USER_IDS = frozenset({LOCAL_USER_ID, DRY_RUN_USER_ID, SINGLE_TENANT_USER_ID})@@ -85,8 +85,7 @@ class RunIdentityPolicy(StrEnum):A run's `user_id` is the host's own identifier for its caller, and itsgroups are the host's own entities, so a stream that identifies people- sends both as they are: `DIRECT`. Pipelex's stream always does. The- operator's stream does unless the operator set `anonymous` mode, which turns+ sends both as they are: `DIRECT`. The operator's stream does unless the operator set `anonymous` mode, which turnsthe whole thing off with `NONE`, because an operator who chose to identifynobody chose it for their users too."""