Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

agent CLI slience

Louis Choquel3a23bffb19b5
diff --git a/TODOS.md b/TODOS.md
index c9d12f8a4..2bd2346a8 100644
--- a/TODOS.md
+++ b/TODOS.md
@@ -1,49 +1,128 @@
-# TODOS
+# Agent CLI logging-discipline hardening — TODOS
-No open work items. This branch (`refactor/ECR`, branched from `feature/Temporal-merge-3` via `feature/Error-handling-2`) is the Extract / Classify / Render decomposition — the final sweep of the error-handling overhaul. What follows is a guide for reviewing it.
+## Context (read this first on a cold start)
-## Reviewing this branch
+**Branch:** `release/v0.30.1`
+**Working dir:** `/Users/lchoquel/repos/Pipelex/pipelex`
-The current-state reference is **`wip/error-handling/`**. Start with [`wip/error-handling/README.md`](wip/error-handling/README.md): its status table and suggested read order map every track to its doc and say what landed vs. what is still only proposed. For the shipped, user-facing version of the same material, see [`docs/under-the-hood/error-model.md`](docs/under-the-hood/error-model.md) — including its "The Uniform Shape — Extract / Classify / Render" section.
+### What this work is about
-This branch is scoped to the ECR refactor. The single most relevant doc is:
+A reviewer flagged a real bug at `pipelex/cli/agent_cli/commands/agent_cli_factory.py:75` (pre-fix line number): the agent CLI's `package_log_levels` override only pinned `pipelex = OFF`, but `deep_update` deep-merges it into the user config. Third-party loggers (anthropic, httpx, botocore, openai, ...) configured at INFO/WARNING in `pipelex/pipelex.toml` kept emitting through the root's `RichHandler` (now pointing at stderr) — corrupting the JSON error envelope that downstream parsers (e.g. `mthds-js`'s `PipelexRunner`) expect.
-- [`wip/error-handling/track-extract-classify-render.md`](wip/error-handling/track-extract-classify-render.md) — what the post-refactor code looks like, plus the design motivation that led to it.
-- [`wip/error-handling/archive-extract-classify-render.md`](wip/error-handling/archive-extract-classify-render.md) — checkpoint history and the key deviations from the original design (e.g. why Classify + Render live in their own modules, why `is_quota_exhaustion` is checked early, why Azure ImgGen keeps two `AMBIGUOUS` branches).
+### What's been done in the current session
-The rest of the error-handling tracks are unchanged on this branch but provide context:
+First pass tried to enumerate every third-party logger (brittle list). User pushed back: "Listing feels like an uphill battle — do we control enough of the logging config to just cut off all logs to stdout and stderr regardless of what package sends them?"
-1. [`architecture.md`](wip/error-handling/architecture.md) — layer model, exception hierarchy, `ErrorReport` shape. Read first; every track refers back to it.
-2. [`track-metadata-model.md`](wip/error-handling/track-metadata-model.md) — the data contract: `error_category`, `error_domain`, `user_action`, `ProviderErrorMetadata` (the type now aliased as `SDKErrorEnvelope` in ECR signatures). Key files: `pipelex/base_exceptions.py`, `pipelex/cogt/exceptions.py`.
-3. [`track-worker-classification.md`](wip/error-handling/track-worker-classification.md) — the per-worker classification sweep that ECR consolidates. Key files: `pipelex/plugins/*/*_worker.py`.
-4. [`track-retry-and-resilience.md`](wip/error-handling/track-retry-and-resilience.md) — explicit transport retry, removal of the `PipeRouter` retry loop, bounded `PipeBatch` fan-out.
-5. [`track-cli-delivery.md`](wip/error-handling/track-cli-delivery.md) — markdown/JSON agent-CLI delivery, `error_domain` → HTTP-status mapping.
-6. [`track-temporal-integration.md`](wip/error-handling/track-temporal-integration.md) — `ErrorReport` carried across the activity → workflow → submitter boundary.
-7. [`track-testing.md`](wip/error-handling/track-testing.md) — cross-cutting; verifies the rest.
+Second pass landed the correct fix: `logging.disable(LOGGING_LEVEL_OFF)` as a process-global cutoff called at the start of `make_pipelex_for_agent_cli` and `agent_doctor_cmd`. Records get filtered in `Logger.isEnabledFor` BEFORE any per-logger level check — no list to maintain.
-## Where the changes are
+Files modified in this session (staged, not committed):
-Production code:
+- `pipelex/cli/agent_cli/commands/agent_cli_factory.py` — added `silence_logging_for_agent_cli()`; simplified `apply_agent_cli_output_discipline()`; simplified `AGENT_CLI_STDERR_LOG_FIELDS` back to `{pipelex: OFF}`
+- `pipelex/cli/agent_cli/commands/doctor_cmd.py` — wired `silence_logging_for_agent_cli` at top of `agent_doctor_cmd`
+- `tests/e2e/agent_cli/test_stdout_is_clean_json.py` — added the third-party-logger leak regression test; refactored `_set_package_log_level` helper
+- `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py` — updated for the new design
+- `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py` — replaced per-logger level assertion with `isEnabledFor` checks
+- `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` — renamed test class, added `silence_logging_for_agent_cli` regression
+- `tests/unit/pipelex/cli/test_agent_doctor_cmd.py` — stubbed `silence_logging_for_agent_cli` in the bootstrap fixture
-- `pipelex/cogt/inference/error_classify.py`, `error_render.py`, `provider_name.py` — new modules. Classify is one pure, provider-blind function; Render picks the `CogtError` subclass from `InferenceErrorFamily` + the `is_model_not_found` flag.
-- `pipelex/cogt/inference/error_classification.py` — `ProviderErrorMetadata` gains `message` + three `@property` accessors (`is_quota_exhaustion`, `is_content_policy_violation`, `is_network_error`). 12 `extract_*_metadata` functions updated to populate `message`. `_AWS_ERROR_CODE_TO_STATUS` map added for Bedrock.
-- 6 LLM workers (`anthropic`, `openai_completions`, `openai_responses`, `mistral`, `google`, `bedrock`) — every `except` block collapsed to `extract → classify → raise render(...) from exc`.
-- 7 img-gen workers (`openai`, `openai_completions_img_gen`, `fal`, `huggingface`, `google_img_gen`, `gateway_img_gen`, `azure_rest`) — same migration. Azure keeps two worker-specific `AMBIGUOUS` branches; see the key deviations in the archive.
-- 7 extract + search workers (`mistral`, `linkup` x2, `gateway` x2, `docling`, `pypdfium2`) — same migration. Mistral + gateway-search now specialize HTTP 404 to `ExtractModelNotFoundError` / `SearchModelNotFoundError`.
-- Deleted: 4 per-provider `*_error_classification.py` files (+ matching tests), `AnthropicCredentialsError`, `GatewayFactory.classify_error_category` / `make_user_action_from_portkey_error` / `make_error_summary_from_portkey_error`, and every inline `_classify_*_error` / `_raise_categorized_*` instance method that the workers used to carry.
+State: `make agent-check` clean. Targeted unit/integration/e2e tests pass (1541 pass / 1 skip).
-Tests:
+### Why we have more TODOs
-- `tests/unit/pipelex/cogt/inference/test_classify_inference_error.py` — provider-blind parity matrix (synthesized envelopes → expected category + user-action kind).
-- `tests/unit/pipelex/cogt/inference/test_render_inference_error.py` — render correctness against synthetic envelopes + classifications.
-- `tests/unit/pipelex/cogt/inference/test_provider_classification_parity.py` — meta-test that walks every `ProviderName` against the extract-fn registry and the worker-family map, so adding a new provider without wiring it fails fast.
-- Per-worker classification tests were updated to the new three-line shape; most categorization assertions have moved to the parity matrix.
+A `/code-review` pass found 15 findings. After triage with the user, 6 actionable items remain (rest are skipped or made moot by the structural fix). Decisions are baked into the steps below.
-## What is intentionally left open
+### Key references
-So these are not flagged as missing during review:
+- The source-of-truth docstring for the cutoff is `pipelex/cli/agent_cli/commands/agent_cli_factory.py:95` (`silence_logging_for_agent_cli`).
+- The comment block at `pipelex/cli/agent_cli/commands/agent_cli_factory.py:32-83` explains why the listing approach was abandoned and how the four knobs in `AGENT_CLI_STDERR_LOG_FIELDS` relate to the global cutoff.
+- The e2e regression test for the original third-party leak: `tests/e2e/agent_cli/test_stdout_is_clean_json.py::test_models_json_stdout_and_stderr_stay_clean_under_third_party_logger_enabled`.
-- **Metadata-model long tail** — a few non-inference `PipelexError` subclasses still rely on the `agent_output.py` fallback dicts for `hint` / `error_domain` instead of class-level metadata. Pre-existing; see the Followups in [`track-metadata-model.md`](wip/error-handling/track-metadata-model.md). Out of scope for ECR.
-- Optional, non-blocking review notes from earlier error-handling sweeps (Temporal activity boundary, Phase 12 search workers) are collected in [`wip/error-handling/review-notes/`](wip/error-handling/review-notes/). Each entry is a deliberate non-fix recorded with a conditional trigger for when to revisit. None are ECR regressions; ECR has if anything reduced their surface by routing the same code paths through the shared parity matrix.
+---
-Completed plans are archived under `wip/error-handling/archive-*.md` — kept for their running notes, not authoritative for the current state. The ECR archive ([`archive-extract-classify-render.md`](wip/error-handling/archive-extract-classify-render.md)) is the new one this branch adds.
+## Phase 1 — Structural fix (the biggest change)
+
+Single architectural shift: every agent CLI command must traverse `app_callback` in `pipelex/cli/agent_cli/_agent_cli.py`. Wiring the silence call there guarantees every current AND future command is covered, even ones (like `init_cmd`, `accept_gateway_terms_cmd`) that bypass `make_pipelex_for_agent_cli`.
+
+- [ ] **Step 1.1** — Read `pipelex/cli/agent_cli/_agent_cli.py` to locate `app_callback` (around line 81 per the review). Confirm it's the typer choke point every command runs through, and that `set_agent_cli_error_format(CliOutputFormat.JSON)` already lives there.
+- [ ] **Step 1.2** — Add `silence_logging_for_agent_cli()` call at the top of `app_callback`, BEFORE `set_agent_cli_error_format`. Import the symbol from `pipelex.cli.agent_cli.commands.agent_cli_factory`.
+- [ ] **Step 1.3** — Decide: keep the existing per-call invocations in `make_pipelex_for_agent_cli:166` and `agent_doctor_cmd:134` OR remove them.
+ - **Decision recorded:** KEEP both. `silence_logging_for_agent_cli` is idempotent (calling `logging.disable` twice with the same value is a no-op). Keeping preserves defense for direct library callers of `make_pipelex_for_agent_cli` and avoids breaking the unit tests that mock `Pipelex.make` and call the factory directly (they assert `logging.root.manager.disable == LOGGING_LEVEL_OFF` after the call).
+- [ ] **Step 1.4** — Update docstrings: `silence_logging_for_agent_cli`'s docstring says "Must be called at the very start of every agent CLI entry point (`make_pipelex_for_agent_cli`, `agent_doctor_cmd`)" — rewrite to reflect that `app_callback` is now the primary armor and the per-call invocations are belt-and-braces.
+
+### ✅ CHECKPOINT 1 — verify before continuing
+
+The structural change is invasive enough that a regression here would mask everything else. STOP here and:
+
+- [ ] Run `make agent-check` — must be clean.
+- [ ] Run the unit suite for cli/: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/`. Must pass.
+- [ ] Run the e2e: `.venv/bin/pytest --tb=short -q tests/e2e/agent_cli/test_stdout_is_clean_json.py`. Must pass — both the original three tests AND the new third-party-leak test.
+- [ ] Manually verify with a fresh shell that `pipelex-agent init --format json` produces a clean stderr in offline mode (no httpx INFO leaks).
+
+**Cold-start handoff at this checkpoint:** if the session ends here, the next agent should re-read this TODOS.md (especially the "Context" section above), then `git diff HEAD` to see what's staged. If Step 1.2 landed but the verify failed, the regression is most likely in `_agent_cli.py` — check that `app_callback` runs unconditionally for every command, not gated on some Typer flag. If verify passed, Phase 2 is independent and can start fresh.
+
+---
+
+## Phase 2 — Production-code hardening + test-fixture leak fix
+
+Three small fixes, tightly related. Land together.
+
+- [ ] **Step 2.1 (fixes #9)** — In `pipelex/cli/agent_cli/commands/agent_cli_factory.py`, change `silence_logging_for_agent_cli` to call `logging.disable(sys.maxsize)` instead of `logging.disable(LOGGING_LEVEL_OFF)`. Add `import sys` if missing. Keep `LOGGING_LEVEL_OFF` import only if still used elsewhere in the file. Update the docstring's "blocks DEBUG through CRITICAL" line to "blocks every record at every level (including custom levels above CRITICAL)".
+- [ ] **Step 2.2 (fixes #9 tests)** — Update assertions in `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py:85` and `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` that compare `logging.root.manager.disable` to `LOGGING_LEVEL_OFF`. Change to compare to `sys.maxsize`. Update imports.
+- [ ] **Step 2.3 (fixes #3)** — In `tests/unit/pipelex/cli/test_agent_cli_factory.py:31-39`, extend the autouse `_restore_globals` fixture to save/restore `logging.root.manager.disable` alongside `PrettyPrinter.mode` and `root_logger.level`. Follow the pattern from `test_agent_cli_factory_init_overrides.py`.
+- [ ] **Step 2.4 (fixes #11)** — In `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py:22-30` AND `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py:32-40`, extend the autouse fixture to snapshot and restore the `.level` attribute of every logger the tests arm. Targets to cover:
+ - In `test_agent_cli_factory_suppression.py`: `pipelex`, `anthropic`, `httpx`, `some.unknown.transitive.dep`.
+ - In `test_agent_cli_output_discipline.py`: `anthropic`, `httpx`, `some.transitive.dep.we.never.heard.of`, AND the loggers derived from `_THIRD_PARTY_PACKAGES` via `package_name.replace("-", ".")`.
+
+### ✅ CHECKPOINT 2 — verify before continuing
+
+Production-code changes are done. Test-only work remains.
+
+- [ ] Run `make agent-check`. Must be clean.
+- [ ] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/test_log_config.py tests/e2e/agent_cli/`. Must pass.
+- [ ] Run `make tb` (boot test) — sanity check on config loading.
+
+**Cold-start handoff at this checkpoint:** next agent reads this TODOS.md, runs `git diff HEAD --stat` to see what's staged. Phase 3 is test-only and can land in a fresh session safely. If anything in Phase 2 broke, the most likely culprits are: (a) the `sys.maxsize` swap left a dangling `LOGGING_LEVEL_OFF` import; (b) the fixture extension overlooked one of the logger names. Re-grep the test files for `setLevel(` calls and verify each target is in the restore loop.
+
+---
+
+## Phase 3 — Test/helper robustness (independent improvements)
+
+- [ ] **Step 3.1 (fixes #13)** — In `tests/e2e/agent_cli/test_stdout_is_clean_json.py::test_models_json_stdout_and_stderr_stay_clean_under_third_party_logger_enabled`, replace the `assert result.stderr == ""` with: stderr must be either empty OR parse as a structured error envelope (a dict with `error` field). This catches log-line leaks (which would not be valid JSON) while tolerating environmental noise like ResourceWarnings. Helper: try `json.loads(result.stderr)` and accept either parse-success-with-envelope-shape or empty.
+- [ ] **Step 3.2 (fixes #7, #8, #15)** — Rewrite `_set_package_log_level` in `tests/e2e/agent_cli/test_stdout_is_clean_json.py` using `tomlkit` (already a project dep — see `init_cmd.py`'s usage). Pseudo:
+ ```python
+ import tomlkit
+ doc = tomlkit.parse(pipelex_toml_path.read_text())
+ doc["pipelex"]["log_config"]["package_log_levels"][package_name] = level
+ pipelex_toml_path.write_text(tomlkit.dumps(doc))
+ ```
+ This eliminates all three line-based-rewriter edges (duplicate section, no-space matcher, no-trailing-newline) in one pass. Keep the `_set_pipelex_package_log_level_to_debug` compat shim so the original two callers don't change.
+- [ ] **Step 3.3** — Sanity check: read the rewritten test file to confirm the section path `["pipelex"]["log_config"]["package_log_levels"]` exists in the kit's `pipelex.toml` (it does, line 177 — `[pipelex.log_config.package_log_levels]`).
+
+### ✅ CHECKPOINT 3 — final verification
+
+- [ ] Run `make agent-check`. Must be clean.
+- [ ] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/ tests/integration/pipelex/cli/ tests/e2e/agent_cli/`. All pass.
+- [ ] Run `make tb`. Pass.
+- [ ] Skim `git diff HEAD --stat` — the diff should now cover: `_agent_cli.py` (new silence wire), `agent_cli_factory.py` (sys.maxsize), `test_agent_cli_factory.py` (fixture), `test_agent_cli_factory_suppression.py` + `test_agent_cli_output_discipline.py` (per-logger restore), `test_stdout_is_clean_json.py` (relaxed assertion + tomlkit rewrite).
+
+---
+
+## Skipped findings (recorded so a future session doesn't re-litigate)
+
+The `/code-review` surfaced 15 findings; these 9 were explicitly skipped after triage:
+
+- **#4 + #5 (warnings module / print_to_stderr leaks)** — Skipped. `logging.disable` doesn't cover these channels (`warnings.warn` → `sys.stderr.write` directly; `print_to_stderr` in `remote_config_fetcher.py:82/219/259`). Doctor path has no `catch_warnings` wrapper, so `GatewayOverrideWarning` (UserWarning subclass) could leak there. Decision: no real-world report; defer until someone hits it. Phase 3 Step 3.1's relaxed e2e assertion absorbs the noise.
+- **#6 (process-global `logging.disable` leaks for in-process callers)** — Skipped. Documented one-shot CLI; embedded/in-process use isn't supported. Could add a docstring note but no functional change.
+- **#10 (apply_agent_cli_output_discipline lost the `pipelex = OFF` backstop)** — Skipped. With `silence_logging_for_agent_cli` in `app_callback` (Phase 1), every path that calls `apply_agent_cli_output_discipline` also has silence armed. Restoring the backstop would be cargo-cult.
+- **#14 (silence ordering in `agent_doctor_cmd`)** — Made moot by Phase 1. `app_callback` runs before any command body, so silence is armed before `set_agent_cli_error_format` runs in `agent_doctor_cmd`.
+
+---
+
+## End-to-end completion criteria
+
+When all phases are checked off:
+
+- [ ] `git diff HEAD --stat` shows the expected files (see Checkpoint 3 list).
+- [ ] All three e2e tests in `test_stdout_is_clean_json.py` pass.
+- [ ] No regression in `make agent-test` (full suite — run before commit per CLAUDE.md release/commit conventions).
+- [ ] Decide whether this work folds into the existing `release/v0.30.1` branch or warrants a follow-up release entry in CHANGELOG. (Note: per memory `feedback_no_unreleased_header.md`, use `[Unreleased]` on work branches — replace with versioned entry only at release time. The current branch is already named `release/v0.30.1` so check with user before bumping.)
diff --git a/pipelex/cli/agent_cli/commands/agent_cli_factory.py b/pipelex/cli/agent_cli/commands/agent_cli_factory.py
index 4724ce0c2..e8818faf3 100644
--- a/pipelex/cli/agent_cli/commands/agent_cli_factory.py
+++ b/pipelex/cli/agent_cli/commands/agent_cli_factory.py
@@ -1,5 +1,6 @@
"""Factory function for agent CLI commands -- JSON-only error output."""
+import logging
import warnings
from collections.abc import Mapping
from pathlib import Path
@@ -25,11 +26,11 @@ from pipelex.system.pipelex_service.exceptions import (
from pipelex.system.runtime import IntegrationMode
from pipelex.system.telemetry.exceptions import TelemetryConfigValidationError
from pipelex.tools.log.log import log
-from pipelex.tools.log.log_levels import LogLevel
+from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
# Canonical leaf dict for "for any agent-CLI invocation, both Rich-managed channels land
-# on stderr AND all pipelex logs are silenced from the very first ``log.configure`` call."
+# on stderr AND Python's logging system is globally muted from the very first init call."
# Consumed by:
# - ``AGENT_CLI_CONFIG_OVERRIDES`` below wraps it in the full-config-tree shape required
# by ``Pipelex.make(config_overrides=...)`` for the full-init path.
@@ -37,20 +38,34 @@ from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
# ``setup_doctor_runtime(log_config_overrides=...)`` for the doctor-only path that
# does not go through ``Pipelex.make``.
#
-# Four knobs:
+# The four knobs in this dict shape Pipelex's own log infrastructure but they CANNOT
+# enumerate every third-party logger a transitive dependency might create. The
+# bulletproof cutoff lives in ``silence_logging_for_agent_cli`` below, which calls
+# ``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold checked inside
+# ``Logger.isEnabledFor`` BEFORE any per-logger level. With that in effect, no record
+# is created for any logger, regardless of which package emits, how it's configured, or
+# what handlers it attaches. We call it as the very first line of
+# ``make_pipelex_for_agent_cli`` and ``agent_doctor_cmd`` so the cutoff is active before
+# any setup code can reach a ``log.*`` call.
+#
+# The dict below stays useful as defense-in-depth + Rich-channel pinning:
# - ``console_log_target`` -> the ``RichHandler`` for Python's logging system
-# (``log.debug/info/warning/error``).
+# (``log.debug/info/warning/error``). Pinned to stderr so
+# if ``logging.disable`` is ever cleared in this process,
+# logs still land on the diagnostic channel rather than
+# corrupting the stdout success envelope.
# - ``console_print_target`` -> the ``Console`` returned by ``get_console()`` and used
# for banners, deck notices, and the main ``pipelex`` CLI's
-# ``show backends`` / ``show models`` tables.
-# - ``default_log_level`` -> root-logger level. Pinned at OFF so any third-party
-# package that inherits root (no explicit override) is
-# fully silenced.
-# - ``package_log_levels`` -> per-package overrides. ``pipelex`` is pinned at OFF here;
-# deep-merged into the user's config, so third-party
-# package levels (anthropic, asyncio, ...) are preserved.
+# ``show backends`` / ``show models`` tables. Rich Console
+# is INDEPENDENT of Python logging — ``logging.disable``
+# does not touch it — so this pin is the only thing
+# keeping Rich tables off stdout.
+# - ``default_log_level`` -> root-logger level. Pinned at OFF as a backup in case
+# ``logging.disable`` is cleared.
+# - ``package_log_levels`` -> historic ``pipelex = OFF`` pin. Redundant under
+# ``logging.disable`` but kept as defense.
#
-# Why silence all pipelex logs? The agent CLI is machine-consumed. Its contract is:
+# Why silence everything? The agent CLI is machine-consumed. Its contract is:
# stdout = structured success envelope (JSON or markdown), stderr = structured error
# envelope. ANY log line on stderr — DEBUG, INFO, WARNING, anything — corrupts the
# error-envelope channel for downstream parsers (e.g. mthds-js's ``PipelexRunner``
@@ -60,9 +75,9 @@ from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
#
# The agent CLI's actual results are written by ``agent_success`` /
# ``agent_success_formatted`` via the bare builtin ``print(...)`` to ``sys.stdout``;
-# error envelopes via ``print(..., file=sys.stderr)``. Both bypass Rich entirely, so
-# the OFF-level pin has no effect on the structured envelopes — only on free-floating
-# ``log.*`` calls scattered through the codebase.
+# error envelopes via ``print(..., file=sys.stderr)``. Both bypass Rich and Python's
+# ``logging`` entirely, so neither ``logging.disable`` nor the pins below affect the
+# structured envelopes themselves.
#
# Wrapped in ``MappingProxyType`` so a stray ``AGENT_CLI_STDERR_LOG_FIELDS[...] = ...`` in
# a future contributor's hot-fix raises immediately instead of silently mutating the
@@ -76,6 +91,31 @@ AGENT_CLI_STDERR_LOG_FIELDS: Mapping[str, Any] = MappingProxyType(
}
)
+
+def silence_logging_for_agent_cli() -> None:
+ """Process-global cutoff of Python's logging system for agent CLI invocations.
+
+ Calls ``logging.disable(LOGGING_LEVEL_OFF)``, which sets
+ ``logging.Logger.manager.disable = LOGGING_LEVEL_OFF``. Every ``Logger.isEnabledFor``
+ call checks ``self.manager.disable >= level`` before the per-logger level check, so
+ no record gets created for any logger at any standard level (DEBUG through CRITICAL)
+ — regardless of which package emits, what level the logger is configured at, or
+ what handlers it has attached. This includes third-party packages we never
+ enumerate and any logger created AFTER this call.
+
+ Idempotent. Must be called at the very start of every agent CLI entry point
+ (``make_pipelex_for_agent_cli``, ``agent_doctor_cmd``) so the cutoff is active
+ before any setup code can reach a ``log.*`` call — closing the window between
+ ``log.configure`` and ``apply_agent_cli_output_discipline``.
+
+ Does NOT affect Rich ``Console`` output (banners, tables, pretty_print) — that's
+ handled by ``console_print_target = STDERR`` + ``PrettyPrinter.mode = SILENT`` +
+ the hub's ``set_console_print_target``. Does NOT affect bare ``print(...)`` calls
+ — those are reserved for the structured success/error envelopes by design.
+ """
+ logging.disable(LOGGING_LEVEL_OFF)
+
+
# Full ``config_overrides`` tree for ``Pipelex.make()``. ``deep_update`` recurses into
# ``package_log_levels``, so the ``pipelex = OFF`` entry merges into the user's config
# without wiping out third-party package levels. Frozen so callers can't mutate it.
@@ -91,22 +131,23 @@ AGENT_CLI_CONFIG_OVERRIDES: Mapping[str, Any] = MappingProxyType(
def apply_agent_cli_output_discipline() -> None:
- """Pin pipelex log level to OFF, silence pretty-print, and hub console target to stderr.
+ """Reaffirm the agent CLI output contract on the Rich/hub channels post-init.
+
+ The bulletproof logging cutoff is ``silence_logging_for_agent_cli`` (called at the
+ start of every agent CLI entry point); this helper handles the channels that are
+ INDEPENDENT of Python's logging system:
- Called from two paths:
- - ``make_pipelex_for_agent_cli`` (full-init): defense-in-depth. ``Pipelex.__init__``
- already pinned the hub console via ``set_console_print_target`` from the loaded
- log_config (whose ``console_print_target`` was overridden to STDERR by
- ``AGENT_CLI_CONFIG_OVERRIDES``, which also pinned the pipelex log level to OFF).
- - ``agent_doctor_cmd`` (doctor-only): also defense-in-depth, since
- ``setup_doctor_runtime`` now mirrors ``Pipelex.__init__`` and applies
- ``set_console_print_target`` itself from the overridden log_config.
+ 1. ``log.redirect_to_stderr`` keeps the RichHandler's console on stderr — defense
+ in case ``logging.disable`` is ever cleared.
+ 2. ``PrettyPrinter.mode = SILENT`` neutralizes ``pretty_print(...)`` entirely
+ (Rich-based, not logging-based).
+ 3. Hub-level ``set_console_print_target(STDERR)`` for the Rich ``Console`` used by
+ banners / tables (also Rich-based, not logging-based).
Safe to call from the broken-config doctor path where ``setup_doctor_runtime`` was
skipped: ``log.redirect_to_stderr`` no-ops when no rich_handler is registered, and
the hub print-target call is gated on a hub being installed.
"""
- log.set_level_for_package("pipelex", LogLevel.OFF)
log.redirect_to_stderr()
PrettyPrinter.mode = PrettyPrintMode.SILENT
hub = PipelexHub.get_optional_instance()
@@ -160,6 +201,9 @@ def make_pipelex_for_agent_cli(
typer.Exit: If initialization fails (after printing JSON error to stderr),
or if inference setup is required (after printing markdown to stdout).
"""
+ # Process-global logging cutoff, BEFORE Pipelex.make can trigger any third-party
+ # log line (anthropic/httpx/botocore credential probes, telemetry setup, etc.).
+ silence_logging_for_agent_cli()
try:
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always", RemoteConfigStaleWarning)
diff --git a/pipelex/cli/agent_cli/commands/doctor_cmd.py b/pipelex/cli/agent_cli/commands/doctor_cmd.py
index 4ec8167ca..870e1b469 100644
--- a/pipelex/cli/agent_cli/commands/doctor_cmd.py
+++ b/pipelex/cli/agent_cli/commands/doctor_cmd.py
@@ -5,7 +5,11 @@ from typing import Annotated, Any
import typer
from pipelex.base_exceptions import PipelexConfigError
-from pipelex.cli.agent_cli.commands.agent_cli_factory import AGENT_CLI_STDERR_LOG_FIELDS, apply_agent_cli_output_discipline
+from pipelex.cli.agent_cli.commands.agent_cli_factory import (
+ AGENT_CLI_STDERR_LOG_FIELDS,
+ apply_agent_cli_output_discipline,
+ silence_logging_for_agent_cli,
+)
from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, agent_success, set_agent_cli_error_format
from pipelex.cli.commands.doctor_cmd import (
BackendFileReport,
@@ -125,6 +129,9 @@ def agent_doctor_cmd(
Use --global/-g to force checking the global ~/.pipelex/ directory.
"""
set_agent_cli_error_format(error_format or output_format)
+ # Process-global logging cutoff, BEFORE setup_doctor_runtime / check_* can trigger
+ # any third-party log line.
+ silence_logging_for_agent_cli()
try:
# When --global, force checking ~/.pipelex/ only; otherwise use layered resolution
config_dir = config_manager.global_config_dir if global_ else None
diff --git a/tests/e2e/agent_cli/test_stdout_is_clean_json.py b/tests/e2e/agent_cli/test_stdout_is_clean_json.py
index b6cfdfe97..a70a9210c 100644
--- a/tests/e2e/agent_cli/test_stdout_is_clean_json.py
+++ b/tests/e2e/agent_cli/test_stdout_is_clean_json.py
@@ -41,33 +41,57 @@ if TYPE_CHECKING:
from pathlib import Path
-def _set_pipelex_package_log_level_to_debug(pipelex_toml_path: Path) -> None:
- """Rewrite the ``pipelex = "INFO"`` line inside ``[pipelex.log_config.package_log_levels]``
- to ``"DEBUG"`` so any setup-time ``log.debug`` in the ``pipelex.*`` namespace fires.
+def _set_package_log_level(pipelex_toml_path: Path, *, package_name: str, level: str) -> None:
+ """Set ``<package_name> = "<level>"`` inside ``[pipelex.log_config.package_log_levels]``.
+
+ If the package key already exists in the section, rewrites its value. Otherwise
+ appends a new line at the end of the section. Either way the user's TOML ends up
+ explicitly carrying the level, which is what we need to exercise the
+ "user-configured third-party logger" leak scenario.
- Targets the specific kit-template structure (``pipelex = "INFO"`` lives directly under
- that section). Asserts on no-match so a future kit refactor surfaces here instead of
+ Asserts on missing-section so a future kit refactor surfaces here instead of
silently neutering the test.
"""
original_text = pipelex_toml_path.read_text(encoding="utf-8")
lines = original_text.splitlines(keepends=True)
+ target_section_start: int | None = None
+ target_section_end: int | None = None
in_target_section = False
- rewrote = False
for index, line in enumerate(lines):
stripped = line.strip()
if stripped.startswith("[") and stripped.endswith("]"):
- in_target_section = stripped == "[pipelex.log_config.package_log_levels]"
- continue
- if in_target_section and stripped.startswith("pipelex"):
- lines[index] = 'pipelex = "DEBUG"\n'
- rewrote = True
- break
- if not rewrote:
- msg = f"Could not find 'pipelex = ...' under [pipelex.log_config.package_log_levels] in {pipelex_toml_path}"
+ if in_target_section:
+ target_section_end = index
+ in_target_section = False
+ if stripped == "[pipelex.log_config.package_log_levels]":
+ in_target_section = True
+ target_section_start = index
+ if in_target_section:
+ target_section_end = len(lines)
+ if target_section_start is None or target_section_end is None:
+ msg = f"Could not find [pipelex.log_config.package_log_levels] in {pipelex_toml_path}"
raise AssertionError(msg)
+
+ new_line = f'{package_name} = "{level}"\n'
+ for index in range(target_section_start + 1, target_section_end):
+ if lines[index].strip().startswith(f"{package_name} "):
+ lines[index] = new_line
+ break
+ else:
+ # Insert before the first trailing blank line so the section's contents stay grouped.
+ insert_index = target_section_end
+ while insert_index > target_section_start + 1 and lines[insert_index - 1].strip() == "":
+ insert_index -= 1
+ lines.insert(insert_index, new_line)
+
pipelex_toml_path.write_text("".join(lines), encoding="utf-8")
+def _set_pipelex_package_log_level_to_debug(pipelex_toml_path: Path) -> None:
+ """Compatibility shim for the original two callers — bumps pipelex to DEBUG."""
+ _set_package_log_level(pipelex_toml_path, package_name="pipelex", level="DEBUG")
+
+
def _set_console_targets(pipelex_toml_path: Path, *, log_target: str, print_target: str) -> None:
"""Rewrite ``console_log_target`` and ``console_print_target`` directly under
``[pipelex.log_config]`` to the given values.
@@ -153,6 +177,58 @@ class TestAgentCliStdoutIsCleanJson:
payload = cast("dict[str, object]", parsed)
assert payload.get("success") is True, f"Expected ``success=True`` in envelope; got {payload!r}"
+ def test_models_json_stdout_and_stderr_stay_clean_under_third_party_logger_enabled(
+ self,
+ hermetic_home: Path,
+ offline_subprocess_env: dict[str, str],
+ ) -> None:
+ """Regression for the third-party-logger leak: even when the user bumps a
+ third-party logger (anthropic, httpx) to DEBUG in their ``pipelex.toml``, the
+ agent CLI's stdout AND stderr channels must stay clean — stdout parseable as a
+ single JSON envelope, stderr free of any log line that would corrupt the
+ structured error envelope.
+
+ Before the fix the agent CLI's ``package_log_levels`` override only pinned
+ ``pipelex = OFF`` and let third-party loggers keep their configured level,
+ leaking INFO/DEBUG records onto stderr via the root's RichHandler. The fix is
+ ``logging.disable(LOGGING_LEVEL_OFF)`` at the start of every agent CLI entry
+ point — a process-global cutoff that blocks every record for every logger,
+ regardless of which package emits.
+ """
+ pipelex_toml = hermetic_home / ".pipelex" / "pipelex.toml"
+ for package_name in ("anthropic", "httpx", "botocore", "openai"):
+ _set_package_log_level(pipelex_toml, package_name=package_name, level="DEBUG")
+
+ result = subprocess.run( # noqa: S603
+ [
+ str(PIPELEX_AGENT_BIN),
+ "models",
+ "--format",
+ "json",
+ ],
+ env=offline_subprocess_env,
+ cwd=str(hermetic_home),
+ capture_output=True,
+ text=True,
+ check=False,
+ timeout=120,
+ )
+
+ assert result.returncode == 0, (
+ f"pipelex-agent models --format json must succeed with third-party loggers at DEBUG.\nstdout={result.stdout!r}\nstderr={result.stderr!r}"
+ )
+ # stdout must parse as JSON in one shot — no log line preceding/following the envelope.
+ parsed = json.loads(result.stdout)
+ assert isinstance(parsed, dict)
+ # stderr must stay empty — any line here proves a third-party logger leaked through
+ # the root RichHandler. Empty-stderr is the strictest possible assertion on the
+ # "no log corrupts the error envelope" contract.
+ assert result.stderr == "", (
+ f"stderr must stay clean even when third-party loggers are bumped to DEBUG — "
+ f"otherwise their records leak through the root's RichHandler and corrupt the "
+ f"JSON error envelope.\nstderr={result.stderr!r}"
+ )
+
def test_models_json_stdout_resists_user_targets_override_to_stdout(
self,
hermetic_home: Path,
diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
index 94fcd67cb..233766103 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
@@ -4,9 +4,11 @@ Regression guard: a user TOML with ``[pipelex.log_config.package_log_levels] pip
"DEBUG"`` used to leak setup-time logs (e.g. ``telemetry_factory.py``'s
``log.debug("Telemetry is disabled...")``, ``validation_error_categorizer.py``'s
``log.warning``) onto stderr — corrupting the JSON error envelope that downstream agent
-consumers parse. The fix pins ``package_log_levels.pipelex = OFF`` via
-``config_overrides`` so suppression takes effect from the very first ``log.configure``
-call inside ``Pipelex.make``, not post-init via ``apply_agent_cli_output_discipline``.
+consumers parse. The bulletproof cutoff is ``silence_logging_for_agent_cli`` which is
+called at the very start of every agent CLI entry point and uses
+``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold that blocks any
+record for any logger, regardless of which package emits or what level it's configured
+at. The ``config_overrides`` carry defense-in-depth pins on top.
The agent CLI has NO ``--log-level`` flag. Suppression is unconditional by design — the
CLI is machine-consumed and stderr is reserved for the structured error envelope.
@@ -14,8 +16,11 @@ CLI is machine-consumed and stderr is reserved for the structured error envelope
from __future__ import annotations
+import logging
from typing import TYPE_CHECKING
+import pytest
+
if TYPE_CHECKING:
from pytest_mock import MockerFixture
@@ -24,25 +29,43 @@ from pipelex.cli.agent_cli.commands.agent_cli_factory import (
make_pipelex_for_agent_cli,
)
from pipelex.system.console_target import ConsoleTarget
-from pipelex.tools.log.log_levels import LogLevel
+from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel
class TestAgentCliFactoryInitOverrides:
- """Pins the contract that init-time logs are suppressed via config_overrides."""
+ """Pins the contract that init-time logs are suppressed via global cutoff + overrides."""
+
+ @pytest.fixture(autouse=True)
+ def _restore_global_logging_disable(self):
+ """``make_pipelex_for_agent_cli`` mutates process-global state via
+ ``logging.disable``. Snapshot the manager's disable level and restore it after
+ each test so the agent CLI cutoff does not leak into other tests in the suite.
+ """
+ original_disable = logging.root.manager.disable
+ yield
+ logging.disable(original_disable)
def test_static_leaf_pins_off_for_pipelex_package(self) -> None:
"""The doctor path and the full-init path BOTH use ``AGENT_CLI_STDERR_LOG_FIELDS``,
- so the leaf itself must silence pipelex logs at OFF.
+ so the leaf itself must silence pipelex logs at OFF as defense-in-depth on top of
+ the global ``logging.disable`` cutoff.
"""
assert AGENT_CLI_STDERR_LOG_FIELDS["default_log_level"] == LogLevel.OFF
assert AGENT_CLI_STDERR_LOG_FIELDS["package_log_levels"]["pipelex"] == LogLevel.OFF
assert AGENT_CLI_STDERR_LOG_FIELDS["console_log_target"] is ConsoleTarget.STDERR
assert AGENT_CLI_STDERR_LOG_FIELDS["console_print_target"] is ConsoleTarget.STDERR
- def test_make_injects_off_overrides_into_pipelex_make(self, mocker: MockerFixture) -> None:
- """``Pipelex.make`` must receive a ``config_overrides`` carrying the OFF pins so
- suppression takes effect at ``log.configure`` time — before any setup-time
- ``log.*`` call can fire on stderr.
+ def test_make_injects_overrides_and_arms_global_logging_cutoff(self, mocker: MockerFixture) -> None:
+ """Two contracts in one call:
+
+ 1. ``Pipelex.make`` must receive a ``config_overrides`` carrying the OFF pins +
+ stderr targets so the Rich channels are correctly aimed before any banner /
+ table / pretty-print fires.
+ 2. ``silence_logging_for_agent_cli`` (called before ``Pipelex.make`` so the
+ cutoff is active during init) must have armed ``logging.disable`` at
+ ``LOGGING_LEVEL_OFF`` — the bulletproof handler-side cutoff that blocks
+ every record for every logger, including third-party libraries we don't
+ enumerate.
"""
mock_pipelex = mocker.MagicMock()
mock_make = mocker.patch(
@@ -58,3 +81,9 @@ class TestAgentCliFactoryInitOverrides:
assert log_config_overrides["package_log_levels"]["pipelex"] is LogLevel.OFF
assert log_config_overrides["console_log_target"] is ConsoleTarget.STDERR
assert log_config_overrides["console_print_target"] is ConsoleTarget.STDERR
+
+ assert logging.root.manager.disable == LOGGING_LEVEL_OFF, (
+ "make_pipelex_for_agent_cli must call silence_logging_for_agent_cli (which arms "
+ "logging.disable at LOGGING_LEVEL_OFF) BEFORE Pipelex.make — otherwise any "
+ "third-party logger configured at INFO/WARNING leaks records during init."
+ )
diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
index 72871e0c3..bd89913fd 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
@@ -11,22 +11,23 @@ if TYPE_CHECKING:
from pytest_mock import MockerFixture
from pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_cli
-from pipelex.tools.log.log_levels import LogLevel
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
class TestAgentCliFactorySuppression:
- """After successful init, the factory should silence pretty-print and lower log verbosity."""
+ """After successful init, the factory should silence pretty-print and Python logging."""
@pytest.fixture(autouse=True)
def _restore_globals(self):
- """Save and restore PrettyPrinter.mode and pipelex log level around each test."""
+ """Save and restore PrettyPrinter.mode and the process-global logging disable
+ threshold around each test so the agent CLI cutoff does not leak into the rest
+ of the suite.
+ """
original_mode = PrettyPrinter.mode
- pipelex_logger = logging.getLogger("pipelex")
- original_level: int = pipelex_logger.level
+ original_disable = logging.root.manager.disable
yield
PrettyPrinter.mode = original_mode
- pipelex_logger.setLevel(original_level)
+ logging.disable(original_disable)
def test_sets_silent_pretty_print_mode(self, mocker: MockerFixture) -> None:
"""PrettyPrinter.mode should be SILENT after make_pipelex_for_agent_cli succeeds."""
@@ -37,15 +38,25 @@ class TestAgentCliFactorySuppression:
assert PrettyPrinter.mode is PrettyPrintMode.SILENT
- def test_silences_pipelex_logger(self, mocker: MockerFixture) -> None:
- """The pipelex package logger must be pinned at OFF (LOGGING_LEVEL_OFF=999) — the
- agent CLI's contract is "no free-floating logs on stderr." There is no log_level
- parameter; suppression is unconditional.
+ def test_silences_every_logger_via_global_logging_disable(self, mocker: MockerFixture) -> None:
+ """The factory must arm ``logging.disable`` so no record can be emitted by any
+ logger — pipelex, anthropic, httpx, botocore, openai, or anything a transitive
+ dependency creates. There is no log_level parameter; suppression is unconditional.
+
+ Checks the observable behavior (``isEnabledFor`` short-circuits to False), not
+ the per-logger level — because under ``logging.disable`` the per-logger level is
+ irrelevant by design.
"""
mock_pipelex = mocker.MagicMock()
mocker.patch("pipelex.cli.agent_cli.commands.agent_cli_factory.Pipelex.make", return_value=mock_pipelex)
+ # Arm a few loggers at INFO/WARNING as a downstream library might.
+ for logger_name in ("pipelex", "anthropic", "httpx", "some.unknown.transitive.dep"):
+ logging.getLogger(logger_name).setLevel(logging.DEBUG)
make_pipelex_for_agent_cli()
- pipelex_logger = logging.getLogger("pipelex")
- assert pipelex_logger.level == LogLevel.OFF.int_logging_level
+ for logger_name in ("pipelex", "anthropic", "httpx", "some.unknown.transitive.dep"):
+ assert not logging.getLogger(logger_name).isEnabledFor(logging.INFO), (
+ f"logger '{logger_name}' must be silenced by the agent CLI global cutoff; "
+ f"any record it emits leaks to stderr and corrupts the JSON error envelope."
+ )
diff --git a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
index 0f3334454..64af8210d 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
@@ -1,11 +1,18 @@
-"""Unit tests for apply_agent_cli_output_discipline real side effects.
-
-The discipline helper is the single source of truth for "agent CLI stdout stays clean."
-The existing test_agent_doctor_cmd suite asserts the helper is called, but the autouse
-fixture there stubs the helper out — so a typo regression inside the helper would ship
-green. This module covers the real side effects (log redirect, PrettyPrinter mode flip,
-hub print-target swap) by patching the dependencies the helper touches and asserting
-the helper drives them correctly.
+"""Unit tests for the agent CLI output discipline functions.
+
+Covers two helpers:
+
+ - ``silence_logging_for_agent_cli`` — process-global ``logging.disable`` cutoff that
+ blocks any record for any logger, regardless of package or level. This is the
+ bulletproof defense against third-party logger leaks (anthropic, httpx, botocore,
+ openai, asyncio, ...) that we can't (and shouldn't try to) enumerate by name.
+ - ``apply_agent_cli_output_discipline`` — pins the Rich/Pretty channels that are
+ INDEPENDENT of Python's logging system (Rich Console for tables/banners,
+ PrettyPrinter for pretty_print, hub-level console target).
+
+The existing test_agent_doctor_cmd suite asserts these helpers are called, but the
+autouse fixture there stubs them out — so a typo regression inside the helper itself
+would ship green. This module covers the real side effects.
"""
from __future__ import annotations
@@ -18,24 +25,54 @@ import pytest
if TYPE_CHECKING:
from pytest_mock import MockerFixture
-from pipelex.cli.agent_cli.commands.agent_cli_factory import apply_agent_cli_output_discipline
+from pipelex.cli.agent_cli.commands.agent_cli_factory import (
+ apply_agent_cli_output_discipline,
+ silence_logging_for_agent_cli,
+)
from pipelex.system.console_target import ConsoleTarget
-from pipelex.tools.log.log_levels import LogLevel
+from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
-class TestApplyAgentCliOutputDiscipline:
+class TestAgentCliOutputDiscipline:
@pytest.fixture(autouse=True)
def _restore_globals(self):
- """Restore PrettyPrinter.mode and the pipelex logger level after each test."""
+ """Restore PrettyPrinter.mode and process-global ``logging.disable`` after each
+ test so the agent CLI cutoff does not leak into other tests in the suite.
+ """
original_mode = PrettyPrinter.mode
- pipelex_logger = logging.getLogger("pipelex")
- original_level = pipelex_logger.level
+ original_disable = logging.root.manager.disable
yield
PrettyPrinter.mode = original_mode
- pipelex_logger.setLevel(original_level)
+ logging.disable(original_disable)
+
+ def test_silence_logging_disables_every_third_party_logger_regardless_of_configured_level(self) -> None:
+ """Regression: ``silence_logging_for_agent_cli`` must call ``logging.disable`` at
+ ``LOGGING_LEVEL_OFF`` so that every logger — pipelex, anthropic, httpx, botocore,
+ openai, anything a transitive dep ever creates — has ``isEnabledFor`` short-circuit
+ to False before its own level check fires. This is the only defense that scales
+ without an enumeration of package names.
+ """
+ # Arm a couple of loggers at INFO/WARNING as if a downstream library had configured
+ # them. The global disable must override their own level checks.
+ anthropic_logger = logging.getLogger("anthropic")
+ anthropic_logger.setLevel(logging.INFO)
+ httpx_logger = logging.getLogger("httpx")
+ httpx_logger.setLevel(logging.WARNING)
+ # A name we never enumerate, to prove the defense is not list-based.
+ random_logger = logging.getLogger("some.transitive.dep.we.never.heard.of")
+ random_logger.setLevel(logging.DEBUG)
+
+ silence_logging_for_agent_cli()
+
+ assert logging.root.manager.disable == LOGGING_LEVEL_OFF
+ assert not anthropic_logger.isEnabledFor(logging.INFO)
+ assert not anthropic_logger.isEnabledFor(logging.CRITICAL)
+ assert not httpx_logger.isEnabledFor(logging.WARNING)
+ assert not random_logger.isEnabledFor(logging.DEBUG)
+ assert not random_logger.isEnabledFor(logging.CRITICAL)
- def test_pins_console_print_target_to_stderr_when_hub_installed(self, mocker: MockerFixture) -> None:
+ def test_apply_discipline_pins_console_print_target_to_stderr_when_hub_installed(self, mocker: MockerFixture) -> None:
"""When a hub is installed, the helper must pin its console_print_target to STDERR."""
mock_redirect = mocker.patch("pipelex.cli.agent_cli.commands.agent_cli_factory.log.redirect_to_stderr")
mock_hub = mocker.MagicMock()
@@ -49,9 +86,8 @@ class TestApplyAgentCliOutputDiscipline:
mock_redirect.assert_called_once()
mock_hub.set_console_print_target.assert_called_once_with(target=ConsoleTarget.STDERR)
assert PrettyPrinter.mode is PrettyPrintMode.SILENT
- assert logging.getLogger("pipelex").level == LogLevel.OFF.int_logging_level
- def test_no_hub_path_skips_hub_call_safely(self, mocker: MockerFixture) -> None:
+ def test_apply_discipline_no_hub_path_skips_hub_call_safely(self, mocker: MockerFixture) -> None:
"""When no hub is installed (broken-config doctor path), the helper must still pin
log + PrettyPrinter without raising — the hub call is gated on get_optional_instance.
"""
@@ -65,4 +101,3 @@ class TestApplyAgentCliOutputDiscipline:
mock_redirect.assert_called_once()
assert PrettyPrinter.mode is PrettyPrintMode.SILENT
- assert logging.getLogger("pipelex").level == LogLevel.OFF.int_logging_level
diff --git a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
index cee11d0e7..f0fcef57d 100644
--- a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
+++ b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
@@ -23,16 +23,20 @@ class TestAgentDoctorCmd:
@pytest.fixture(autouse=True)
def _mock_doctor_bootstrap(self, mocker: MockerFixture) -> None:
- """Stub the runtime bootstrap so unit tests don't load real config or reconfigure logging.
+ """Stub the runtime bootstrap so unit tests don't load real config or mutate
+ global logging / PrettyPrinter state.
``setup_doctor_runtime`` instantiates a PipelexHub, loads config from disk, and
calls ``log.configure`` (once-per-process). ``apply_agent_cli_output_discipline``
- mutates global PrettyPrinter mode and the hub's console target. Both are out of
- scope for these tests — they cover the command's output shape, not the runtime
- bootstrap mechanics.
+ mutates global PrettyPrinter mode and the hub's console target.
+ ``silence_logging_for_agent_cli`` arms ``logging.disable`` at LOGGING_LEVEL_OFF
+ — process-global and would leak into other tests in the suite. All three are
+ out of scope for these tests — they cover the command's output shape, not the
+ runtime bootstrap mechanics.
"""
mocker.patch("pipelex.cli.agent_cli.commands.doctor_cmd.setup_doctor_runtime")
mocker.patch("pipelex.cli.agent_cli.commands.doctor_cmd.apply_agent_cli_output_discipline")
+ mocker.patch("pipelex.cli.agent_cli.commands.doctor_cmd.silence_logging_for_agent_cli")
def test_healthy_output_json(self, mocker: MockerFixture, capsys: pytest.CaptureFixture[str]) -> None:
"""Healthy checks should produce JSON with all_healthy=true and no recommended_actions."""