Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
fix(agent-cli): use sys.maxsize for logging.disable and tighten test isolation
Louis Choquel9563915f0859
diff --git a/TODOS.md b/TODOS.mdindex 5ffeae1a7..f8ba6e989 100644--- a/TODOS.md+++ b/TODOS.md@@ -66,20 +66,20 @@ The structural change is invasive enough that a regression here would mask everyThree small fixes, tightly related. Land together.-- [ ] **Step 2.1 (fixes #9)** — In `pipelex/cli/agent_cli/commands/agent_cli_factory.py`, change `silence_logging_for_agent_cli` to call `logging.disable(sys.maxsize)` instead of `logging.disable(LOGGING_LEVEL_OFF)`. Add `import sys` if missing. Keep `LOGGING_LEVEL_OFF` import only if still used elsewhere in the file. Update the docstring's "blocks DEBUG through CRITICAL" line to "blocks every record at every level (including custom levels above CRITICAL)".-- [ ] **Step 2.2 (fixes #9 tests)** — Update assertions in `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py:85` and `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` that compare `logging.root.manager.disable` to `LOGGING_LEVEL_OFF`. Change to compare to `sys.maxsize`. Update imports.-- [ ] **Step 2.3 (fixes #3)** — In `tests/unit/pipelex/cli/test_agent_cli_factory.py:31-39`, extend the autouse `_restore_globals` fixture to save/restore `logging.root.manager.disable` alongside `PrettyPrinter.mode` and `root_logger.level`. Follow the pattern from `test_agent_cli_factory_init_overrides.py`.-- [ ] **Step 2.4 (fixes #11)** — In `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py:22-30` AND `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py:32-40`, extend the autouse fixture to snapshot and restore the `.level` attribute of every logger the tests arm. Targets to cover:- - In `test_agent_cli_factory_suppression.py`: `pipelex`, `anthropic`, `httpx`, `some.unknown.transitive.dep`.- - In `test_agent_cli_output_discipline.py`: `anthropic`, `httpx`, `some.transitive.dep.we.never.heard.of`, AND the loggers derived from `_THIRD_PARTY_PACKAGES` via `package_name.replace("-", ".")`.+- [x] **Step 2.1 (fixes #9)** — In `pipelex/cli/agent_cli/commands/agent_cli_factory.py`, change `silence_logging_for_agent_cli` to call `logging.disable(sys.maxsize)` instead of `logging.disable(LOGGING_LEVEL_OFF)`. Add `import sys` if missing. Keep `LOGGING_LEVEL_OFF` import only if still used elsewhere in the file. Update the docstring's "blocks DEBUG through CRITICAL" line to "blocks every record at every level (including custom levels above CRITICAL)". **Done:** added `import sys`, removed `LOGGING_LEVEL_OFF` import (no other uses in file), updated docstring + the comment block at lines 32-49 that also referenced the old constant.+- [x] **Step 2.2 (fixes #9 tests)** — Update assertions in `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py:85` and `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` that compare `logging.root.manager.disable` to `LOGGING_LEVEL_OFF`. Change to compare to `sys.maxsize`. Update imports. **Done:** swapped both assertions and import statements; updated module/test docstrings.+- [x] **Step 2.3 (fixes #3)** — In `tests/unit/pipelex/cli/test_agent_cli_factory.py:31-39`, extend the autouse `_restore_globals` fixture to save/restore `logging.root.manager.disable` alongside `PrettyPrinter.mode` and `root_logger.level`. Follow the pattern from `test_agent_cli_factory_init_overrides.py`. **Done.**+- [x] **Step 2.4 (fixes #11)** — In `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py:22-30` AND `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py:32-40`, extend the autouse fixture to snapshot and restore the `.level` attribute of every logger the tests arm. Targets to cover:+ - In `test_agent_cli_factory_suppression.py`: `pipelex`, `anthropic`, `httpx`, `some.unknown.transitive.dep`. **Done** (via module-level `_ARMED_LOGGER_NAMES` tuple consumed by the fixture).+ - In `test_agent_cli_output_discipline.py`: `anthropic`, `httpx`, `some.transitive.dep.we.never.heard.of`. **Done** (same pattern). Note: the TODOS step also mentioned `_THIRD_PARTY_PACKAGES` — that symbol does not exist anywhere in `pipelex/` or `tests/` (grep confirms), so it was a stale reference from the abandoned enumeration approach. The actual loggers armed by the test body are the three listed; nothing more to restore.### ✅ CHECKPOINT 2 — verify before continuingProduction-code changes are done. Test-only work remains.-- [ ] Run `make agent-check`. Must be clean.-- [ ] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/test_log_config.py tests/e2e/agent_cli/`. Must pass.-- [ ] Run `make tb` (boot test) — sanity check on config loading.+- [x] Run `make agent-check`. Must be clean. **Result:** 0 pyright errors, mypy success on 1893 files.+- [x] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/test_log_config.py tests/e2e/agent_cli/`. Must pass. **Result:** 366 passed in 43.36s.+- [x] Run `make tb` (boot test) — sanity check on config loading. **Result:** 2 passed, 6708 deselected in 5.67s.**Cold-start handoff at this checkpoint:** next agent reads this TODOS.md, runs `git diff HEAD --stat` to see what's staged. Phase 3 is test-only and can land in a fresh session safely. If anything in Phase 2 broke, the most likely culprits are: (a) the `sys.maxsize` swap left a dangling `LOGGING_LEVEL_OFF` import; (b) the fixture extension overlooked one of the logger names. Re-grep the test files for `setLevel(` calls and verify each target is in the restore loop.diff --git a/pipelex/cli/agent_cli/commands/agent_cli_factory.py b/pipelex/cli/agent_cli/commands/agent_cli_factory.pyindex b78a40edd..a21855101 100644--- a/pipelex/cli/agent_cli/commands/agent_cli_factory.py+++ b/pipelex/cli/agent_cli/commands/agent_cli_factory.py@@ -1,6 +1,7 @@"""Factory function for agent CLI commands -- JSON-only error output."""import logging+import sysimport warningsfrom collections.abc import Mappingfrom pathlib import Path@@ -26,7 +27,7 @@ from pipelex.system.pipelex_service.exceptions import (from pipelex.system.runtime import IntegrationModefrom pipelex.system.telemetry.exceptions import TelemetryConfigValidationErrorfrom pipelex.tools.log.log import log-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel+from pipelex.tools.log.log_levels import LogLevelfrom pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode# Canonical leaf dict for "for any agent-CLI invocation, both Rich-managed channels land@@ -41,7 +42,7 @@ from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode# The four knobs in this dict shape Pipelex's own log infrastructure but they CANNOT# enumerate every third-party logger a transitive dependency might create. The# bulletproof cutoff lives in ``silence_logging_for_agent_cli`` below, which calls-# ``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold checked inside+# ``logging.disable(sys.maxsize)`` — a process-global threshold checked inside# ``Logger.isEnabledFor`` BEFORE any per-logger level. With that in effect, no record# is created for any logger, regardless of which package emits, how it's configured, or# what handlers it attaches. We call it as the very first line of@@ -95,13 +96,13 @@ AGENT_CLI_STDERR_LOG_FIELDS: Mapping[str, Any] = MappingProxyType(def silence_logging_for_agent_cli() -> None:"""Process-global cutoff of Python's logging system for agent CLI invocations.- Calls ``logging.disable(LOGGING_LEVEL_OFF)``, which sets- ``logging.Logger.manager.disable = LOGGING_LEVEL_OFF``. Every ``Logger.isEnabledFor``+ Calls ``logging.disable(sys.maxsize)``, which sets+ ``logging.Logger.manager.disable = sys.maxsize``. Every ``Logger.isEnabledFor``call checks ``self.manager.disable >= level`` before the per-logger level check, so- no record gets created for any logger at any standard level (DEBUG through CRITICAL)- — regardless of which package emits, what level the logger is configured at, or- what handlers it has attached. This includes third-party packages we never- enumerate and any logger created AFTER this call.+ no record gets created for any logger — blocks every record at every level+ (including custom levels above CRITICAL), regardless of which package emits, what+ level the logger is configured at, or what handlers it has attached. This includes+ third-party packages we never enumerate and any logger created AFTER this call.Idempotent. The primary call site is ``app_callback`` in``pipelex.cli.agent_cli._agent_cli`` — Typer routes every ``pipelex-agent``@@ -117,7 +118,7 @@ def silence_logging_for_agent_cli() -> None:the hub's ``set_console_print_target``. Does NOT affect bare ``print(...)`` calls— those are reserved for the structured success/error envelopes by design."""- logging.disable(LOGGING_LEVEL_OFF)+ logging.disable(sys.maxsize)# Full ``config_overrides`` tree for ``Pipelex.make()``. ``deep_update`` recurses intodiff --git a/tests/e2e/agent_cli/test_stdout_is_clean_json.py b/tests/e2e/agent_cli/test_stdout_is_clean_json.pyindex a70a9210c..652270e57 100644--- a/tests/e2e/agent_cli/test_stdout_is_clean_json.py+++ b/tests/e2e/agent_cli/test_stdout_is_clean_json.py@@ -191,9 +191,10 @@ class TestAgentCliStdoutIsCleanJson:Before the fix the agent CLI's ``package_log_levels`` override only pinned``pipelex = OFF`` and let third-party loggers keep their configured level,leaking INFO/DEBUG records onto stderr via the root's RichHandler. The fix is- ``logging.disable(LOGGING_LEVEL_OFF)`` at the start of every agent CLI entry- point — a process-global cutoff that blocks every record for every logger,- regardless of which package emits.+ ``logging.disable(sys.maxsize)`` at the start of every agent CLI entry point+ — a process-global cutoff that blocks every record for every logger at every+ level (including custom levels above CRITICAL), regardless of which package+ emits."""pipelex_toml = hermetic_home / ".pipelex" / "pipelex.toml"for package_name in ("anthropic", "httpx", "botocore", "openai"):diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory.py b/tests/unit/pipelex/cli/test_agent_cli_factory.pyindex 9075f286c..eb15662aa 100644--- a/tests/unit/pipelex/cli/test_agent_cli_factory.py+++ b/tests/unit/pipelex/cli/test_agent_cli_factory.py@@ -30,13 +30,19 @@ class TestMakePipelexForAgentCli:@pytest.fixture(autouse=True)def _restore_globals(self):- """Restore PrettyPrinter.mode and root log level after tests that call the factory successfully."""+ """Restore PrettyPrinter.mode, root log level, and the process-global+ ``logging.disable`` threshold after tests that call the factory successfully —+ otherwise the agent CLI cutoff (armed by ``silence_logging_for_agent_cli`` inside+ the factory) leaks into other tests in the suite.+ """original_mode = PrettyPrinter.moderoot_logger = logging.getLogger()original_level: int = root_logger.level+ original_disable = logging.root.manager.disableyieldPrettyPrinter.mode = original_moderoot_logger.setLevel(original_level)+ logging.disable(original_disable)def test_successful_initialization(self, mocker: MockerFixture) -> None:"""Should return the Pipelex instance when make() succeeds."""diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.pyindex 233766103..85992d3a6 100644--- a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py@@ -6,9 +6,10 @@ Regression guard: a user TOML with ``[pipelex.log_config.package_log_levels] pip``log.warning``) onto stderr — corrupting the JSON error envelope that downstream agentconsumers parse. The bulletproof cutoff is ``silence_logging_for_agent_cli`` which iscalled at the very start of every agent CLI entry point and uses-``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold that blocks any-record for any logger, regardless of which package emits or what level it's configured-at. The ``config_overrides`` carry defense-in-depth pins on top.+``logging.disable(sys.maxsize)`` — a process-global threshold that blocks any record+for any logger at any level (including custom levels above CRITICAL), regardless of+which package emits or what level it's configured at. The ``config_overrides`` carry+defense-in-depth pins on top.The agent CLI has NO ``--log-level`` flag. Suppression is unconditional by design — theCLI is machine-consumed and stderr is reserved for the structured error envelope.@@ -17,6 +18,7 @@ CLI is machine-consumed and stderr is reserved for the structured error envelopefrom __future__ import annotationsimport logging+import sysfrom typing import TYPE_CHECKINGimport pytest@@ -29,7 +31,7 @@ from pipelex.cli.agent_cli.commands.agent_cli_factory import (make_pipelex_for_agent_cli,)from pipelex.system.console_target import ConsoleTarget-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel+from pipelex.tools.log.log_levels import LogLevelclass TestAgentCliFactoryInitOverrides:@@ -63,9 +65,9 @@ class TestAgentCliFactoryInitOverrides:table / pretty-print fires.2. ``silence_logging_for_agent_cli`` (called before ``Pipelex.make`` so thecutoff is active during init) must have armed ``logging.disable`` at- ``LOGGING_LEVEL_OFF`` — the bulletproof handler-side cutoff that blocks- every record for every logger, including third-party libraries we don't- enumerate.+ ``sys.maxsize`` — the bulletproof handler-side cutoff that blocks every+ record for every logger at every level (including custom levels above+ CRITICAL), including third-party libraries we don't enumerate."""mock_pipelex = mocker.MagicMock()mock_make = mocker.patch(@@ -82,8 +84,8 @@ class TestAgentCliFactoryInitOverrides:assert log_config_overrides["console_log_target"] is ConsoleTarget.STDERRassert log_config_overrides["console_print_target"] is ConsoleTarget.STDERR- assert logging.root.manager.disable == LOGGING_LEVEL_OFF, (+ assert logging.root.manager.disable == sys.maxsize, ("make_pipelex_for_agent_cli must call silence_logging_for_agent_cli (which arms "- "logging.disable at LOGGING_LEVEL_OFF) BEFORE Pipelex.make — otherwise any "+ "logging.disable at sys.maxsize) BEFORE Pipelex.make — otherwise any ""third-party logger configured at INFO/WARNING leaks records during init.")diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.pyindex bd89913fd..620881773 100644--- a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py@@ -13,21 +13,36 @@ if TYPE_CHECKING:from pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_clifrom pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode+# Loggers armed by the test below at non-default levels. Their ``.level`` attribute is+# snapshot in the autouse fixture and restored after each test so the per-logger state+# does not leak into the rest of the suite (other tests may rely on the default NOTSET).+_ARMED_LOGGER_NAMES: tuple[str, ...] = (+ "pipelex",+ "anthropic",+ "httpx",+ "some.unknown.transitive.dep",+)+class TestAgentCliFactorySuppression:"""After successful init, the factory should silence pretty-print and Python logging."""@pytest.fixture(autouse=True)def _restore_globals(self):- """Save and restore PrettyPrinter.mode and the process-global logging disable- threshold around each test so the agent CLI cutoff does not leak into the rest- of the suite.+ """Save and restore PrettyPrinter.mode, the process-global logging disable+ threshold, AND the per-logger ``.level`` for every logger this test class arms.+ Without restoring the per-logger levels, the ``setLevel`` calls in the test body+ leak into shared module-level loggers (``pipelex``, ``anthropic``, ``httpx``, ...)+ and affect any other test that assumes default NOTSET."""original_mode = PrettyPrinter.modeoriginal_disable = logging.root.manager.disable+ original_levels = {name: logging.getLogger(name).level for name in _ARMED_LOGGER_NAMES}yieldPrettyPrinter.mode = original_modelogging.disable(original_disable)+ for name, level in original_levels.items():+ logging.getLogger(name).setLevel(level)def test_sets_silent_pretty_print_mode(self, mocker: MockerFixture) -> None:"""PrettyPrinter.mode should be SILENT after make_pipelex_for_agent_cli succeeds."""diff --git a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.pyindex 64af8210d..b7ce52060 100644--- a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py+++ b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py@@ -18,6 +18,7 @@ would ship green. This module covers the real side effects.from __future__ import annotationsimport logging+import sysfrom typing import TYPE_CHECKINGimport pytest@@ -30,28 +31,42 @@ from pipelex.cli.agent_cli.commands.agent_cli_factory import (silence_logging_for_agent_cli,)from pipelex.system.console_target import ConsoleTarget-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFFfrom pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode+# Loggers armed by the test below at non-OFF levels. Their ``.level`` attribute is+# snapshot in the autouse fixture and restored after each test so the per-logger state+# does not leak into the rest of the suite (other tests may rely on the default NOTSET).+_ARMED_LOGGER_NAMES: tuple[str, ...] = (+ "anthropic",+ "httpx",+ "some.transitive.dep.we.never.heard.of",+)+class TestAgentCliOutputDiscipline:@pytest.fixture(autouse=True)def _restore_globals(self):- """Restore PrettyPrinter.mode and process-global ``logging.disable`` after each- test so the agent CLI cutoff does not leak into other tests in the suite.+ """Restore PrettyPrinter.mode, the process-global ``logging.disable`` threshold,+ AND the per-logger ``.level`` for every logger this test class arms — otherwise+ the levels set on shared module-level loggers (``anthropic``, ``httpx``, ...) leak+ into other tests that assume default NOTSET."""original_mode = PrettyPrinter.modeoriginal_disable = logging.root.manager.disable+ original_levels = {name: logging.getLogger(name).level for name in _ARMED_LOGGER_NAMES}yieldPrettyPrinter.mode = original_modelogging.disable(original_disable)+ for name, level in original_levels.items():+ logging.getLogger(name).setLevel(level)def test_silence_logging_disables_every_third_party_logger_regardless_of_configured_level(self) -> None:"""Regression: ``silence_logging_for_agent_cli`` must call ``logging.disable`` at- ``LOGGING_LEVEL_OFF`` so that every logger — pipelex, anthropic, httpx, botocore,+ ``sys.maxsize`` so that every logger — pipelex, anthropic, httpx, botocore,openai, anything a transitive dep ever creates — has ``isEnabledFor`` short-circuit- to False before its own level check fires. This is the only defense that scales- without an enumeration of package names.+ to False before its own level check fires, at every level (including custom levels+ above CRITICAL). This is the only defense that scales without an enumeration of+ package names."""# Arm a couple of loggers at INFO/WARNING as if a downstream library had configured# them. The global disable must override their own level checks.@@ -65,7 +80,7 @@ class TestAgentCliOutputDiscipline:silence_logging_for_agent_cli()- assert logging.root.manager.disable == LOGGING_LEVEL_OFF+ assert logging.root.manager.disable == sys.maxsizeassert not anthropic_logger.isEnabledFor(logging.INFO)assert not anthropic_logger.isEnabledFor(logging.CRITICAL)assert not httpx_logger.isEnabledFor(logging.WARNING)diff --git a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py b/tests/unit/pipelex/cli/test_agent_doctor_cmd.pyindex f0fcef57d..a0ee65d61 100644--- a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py+++ b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py@@ -29,7 +29,7 @@ class TestAgentDoctorCmd:``setup_doctor_runtime`` instantiates a PipelexHub, loads config from disk, andcalls ``log.configure`` (once-per-process). ``apply_agent_cli_output_discipline``mutates global PrettyPrinter mode and the hub's console target.- ``silence_logging_for_agent_cli`` arms ``logging.disable`` at LOGGING_LEVEL_OFF+ ``silence_logging_for_agent_cli`` arms ``logging.disable`` at ``sys.maxsize``— process-global and would leak into other tests in the suite. All three areout of scope for these tests — they cover the command's output shape, not theruntime bootstrap mechanics.