Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix(agent-cli): use sys.maxsize for logging.disable and tighten test isolation

Louis Choquel9563915f0859
diff --git a/TODOS.md b/TODOS.md
index 5ffeae1a7..f8ba6e989 100644
--- a/TODOS.md
+++ b/TODOS.md
@@ -66,20 +66,20 @@ The structural change is invasive enough that a regression here would mask every
Three small fixes, tightly related. Land together.
-- [ ] **Step 2.1 (fixes #9)** — In `pipelex/cli/agent_cli/commands/agent_cli_factory.py`, change `silence_logging_for_agent_cli` to call `logging.disable(sys.maxsize)` instead of `logging.disable(LOGGING_LEVEL_OFF)`. Add `import sys` if missing. Keep `LOGGING_LEVEL_OFF` import only if still used elsewhere in the file. Update the docstring's "blocks DEBUG through CRITICAL" line to "blocks every record at every level (including custom levels above CRITICAL)".
-- [ ] **Step 2.2 (fixes #9 tests)** — Update assertions in `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py:85` and `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` that compare `logging.root.manager.disable` to `LOGGING_LEVEL_OFF`. Change to compare to `sys.maxsize`. Update imports.
-- [ ] **Step 2.3 (fixes #3)** — In `tests/unit/pipelex/cli/test_agent_cli_factory.py:31-39`, extend the autouse `_restore_globals` fixture to save/restore `logging.root.manager.disable` alongside `PrettyPrinter.mode` and `root_logger.level`. Follow the pattern from `test_agent_cli_factory_init_overrides.py`.
-- [ ] **Step 2.4 (fixes #11)** — In `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py:22-30` AND `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py:32-40`, extend the autouse fixture to snapshot and restore the `.level` attribute of every logger the tests arm. Targets to cover:
- - In `test_agent_cli_factory_suppression.py`: `pipelex`, `anthropic`, `httpx`, `some.unknown.transitive.dep`.
- - In `test_agent_cli_output_discipline.py`: `anthropic`, `httpx`, `some.transitive.dep.we.never.heard.of`, AND the loggers derived from `_THIRD_PARTY_PACKAGES` via `package_name.replace("-", ".")`.
+- [x] **Step 2.1 (fixes #9)** — In `pipelex/cli/agent_cli/commands/agent_cli_factory.py`, change `silence_logging_for_agent_cli` to call `logging.disable(sys.maxsize)` instead of `logging.disable(LOGGING_LEVEL_OFF)`. Add `import sys` if missing. Keep `LOGGING_LEVEL_OFF` import only if still used elsewhere in the file. Update the docstring's "blocks DEBUG through CRITICAL" line to "blocks every record at every level (including custom levels above CRITICAL)". **Done:** added `import sys`, removed `LOGGING_LEVEL_OFF` import (no other uses in file), updated docstring + the comment block at lines 32-49 that also referenced the old constant.
+- [x] **Step 2.2 (fixes #9 tests)** — Update assertions in `tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py:85` and `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py` that compare `logging.root.manager.disable` to `LOGGING_LEVEL_OFF`. Change to compare to `sys.maxsize`. Update imports. **Done:** swapped both assertions and import statements; updated module/test docstrings.
+- [x] **Step 2.3 (fixes #3)** — In `tests/unit/pipelex/cli/test_agent_cli_factory.py:31-39`, extend the autouse `_restore_globals` fixture to save/restore `logging.root.manager.disable` alongside `PrettyPrinter.mode` and `root_logger.level`. Follow the pattern from `test_agent_cli_factory_init_overrides.py`. **Done.**
+- [x] **Step 2.4 (fixes #11)** — In `tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py:22-30` AND `tests/unit/pipelex/cli/test_agent_cli_output_discipline.py:32-40`, extend the autouse fixture to snapshot and restore the `.level` attribute of every logger the tests arm. Targets to cover:
+ - In `test_agent_cli_factory_suppression.py`: `pipelex`, `anthropic`, `httpx`, `some.unknown.transitive.dep`. **Done** (via module-level `_ARMED_LOGGER_NAMES` tuple consumed by the fixture).
+ - In `test_agent_cli_output_discipline.py`: `anthropic`, `httpx`, `some.transitive.dep.we.never.heard.of`. **Done** (same pattern). Note: the TODOS step also mentioned `_THIRD_PARTY_PACKAGES` — that symbol does not exist anywhere in `pipelex/` or `tests/` (grep confirms), so it was a stale reference from the abandoned enumeration approach. The actual loggers armed by the test body are the three listed; nothing more to restore.
### ✅ CHECKPOINT 2 — verify before continuing
Production-code changes are done. Test-only work remains.
-- [ ] Run `make agent-check`. Must be clean.
-- [ ] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/test_log_config.py tests/e2e/agent_cli/`. Must pass.
-- [ ] Run `make tb` (boot test) — sanity check on config loading.
+- [x] Run `make agent-check`. Must be clean. **Result:** 0 pyright errors, mypy success on 1893 files.
+- [x] Run targeted: `.venv/bin/pytest -n auto -m "(dry_runnable or not (inference or llm or img_gen or extract or search)) and not pipelex_api" -o log_level=WARNING --tb=short -q tests/unit/pipelex/cli/ tests/unit/pipelex/tools/test_log_config.py tests/e2e/agent_cli/`. Must pass. **Result:** 366 passed in 43.36s.
+- [x] Run `make tb` (boot test) — sanity check on config loading. **Result:** 2 passed, 6708 deselected in 5.67s.
**Cold-start handoff at this checkpoint:** next agent reads this TODOS.md, runs `git diff HEAD --stat` to see what's staged. Phase 3 is test-only and can land in a fresh session safely. If anything in Phase 2 broke, the most likely culprits are: (a) the `sys.maxsize` swap left a dangling `LOGGING_LEVEL_OFF` import; (b) the fixture extension overlooked one of the logger names. Re-grep the test files for `setLevel(` calls and verify each target is in the restore loop.
diff --git a/pipelex/cli/agent_cli/commands/agent_cli_factory.py b/pipelex/cli/agent_cli/commands/agent_cli_factory.py
index b78a40edd..a21855101 100644
--- a/pipelex/cli/agent_cli/commands/agent_cli_factory.py
+++ b/pipelex/cli/agent_cli/commands/agent_cli_factory.py
@@ -1,6 +1,7 @@
"""Factory function for agent CLI commands -- JSON-only error output."""
import logging
+import sys
import warnings
from collections.abc import Mapping
from pathlib import Path
@@ -26,7 +27,7 @@ from pipelex.system.pipelex_service.exceptions import (
from pipelex.system.runtime import IntegrationMode
from pipelex.system.telemetry.exceptions import TelemetryConfigValidationError
from pipelex.tools.log.log import log
-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel
+from pipelex.tools.log.log_levels import LogLevel
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
# Canonical leaf dict for "for any agent-CLI invocation, both Rich-managed channels land
@@ -41,7 +42,7 @@ from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
# The four knobs in this dict shape Pipelex's own log infrastructure but they CANNOT
# enumerate every third-party logger a transitive dependency might create. The
# bulletproof cutoff lives in ``silence_logging_for_agent_cli`` below, which calls
-# ``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold checked inside
+# ``logging.disable(sys.maxsize)`` — a process-global threshold checked inside
# ``Logger.isEnabledFor`` BEFORE any per-logger level. With that in effect, no record
# is created for any logger, regardless of which package emits, how it's configured, or
# what handlers it attaches. We call it as the very first line of
@@ -95,13 +96,13 @@ AGENT_CLI_STDERR_LOG_FIELDS: Mapping[str, Any] = MappingProxyType(
def silence_logging_for_agent_cli() -> None:
"""Process-global cutoff of Python's logging system for agent CLI invocations.
- Calls ``logging.disable(LOGGING_LEVEL_OFF)``, which sets
- ``logging.Logger.manager.disable = LOGGING_LEVEL_OFF``. Every ``Logger.isEnabledFor``
+ Calls ``logging.disable(sys.maxsize)``, which sets
+ ``logging.Logger.manager.disable = sys.maxsize``. Every ``Logger.isEnabledFor``
call checks ``self.manager.disable >= level`` before the per-logger level check, so
- no record gets created for any logger at any standard level (DEBUG through CRITICAL)
- — regardless of which package emits, what level the logger is configured at, or
- what handlers it has attached. This includes third-party packages we never
- enumerate and any logger created AFTER this call.
+ no record gets created for any logger — blocks every record at every level
+ (including custom levels above CRITICAL), regardless of which package emits, what
+ level the logger is configured at, or what handlers it has attached. This includes
+ third-party packages we never enumerate and any logger created AFTER this call.
Idempotent. The primary call site is ``app_callback`` in
``pipelex.cli.agent_cli._agent_cli`` — Typer routes every ``pipelex-agent``
@@ -117,7 +118,7 @@ def silence_logging_for_agent_cli() -> None:
the hub's ``set_console_print_target``. Does NOT affect bare ``print(...)`` calls
— those are reserved for the structured success/error envelopes by design.
"""
- logging.disable(LOGGING_LEVEL_OFF)
+ logging.disable(sys.maxsize)
# Full ``config_overrides`` tree for ``Pipelex.make()``. ``deep_update`` recurses into
diff --git a/tests/e2e/agent_cli/test_stdout_is_clean_json.py b/tests/e2e/agent_cli/test_stdout_is_clean_json.py
index a70a9210c..652270e57 100644
--- a/tests/e2e/agent_cli/test_stdout_is_clean_json.py
+++ b/tests/e2e/agent_cli/test_stdout_is_clean_json.py
@@ -191,9 +191,10 @@ class TestAgentCliStdoutIsCleanJson:
Before the fix the agent CLI's ``package_log_levels`` override only pinned
``pipelex = OFF`` and let third-party loggers keep their configured level,
leaking INFO/DEBUG records onto stderr via the root's RichHandler. The fix is
- ``logging.disable(LOGGING_LEVEL_OFF)`` at the start of every agent CLI entry
- point — a process-global cutoff that blocks every record for every logger,
- regardless of which package emits.
+ ``logging.disable(sys.maxsize)`` at the start of every agent CLI entry point
+ — a process-global cutoff that blocks every record for every logger at every
+ level (including custom levels above CRITICAL), regardless of which package
+ emits.
"""
pipelex_toml = hermetic_home / ".pipelex" / "pipelex.toml"
for package_name in ("anthropic", "httpx", "botocore", "openai"):
diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory.py b/tests/unit/pipelex/cli/test_agent_cli_factory.py
index 9075f286c..eb15662aa 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_factory.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_factory.py
@@ -30,13 +30,19 @@ class TestMakePipelexForAgentCli:
@pytest.fixture(autouse=True)
def _restore_globals(self):
- """Restore PrettyPrinter.mode and root log level after tests that call the factory successfully."""
+ """Restore PrettyPrinter.mode, root log level, and the process-global
+ ``logging.disable`` threshold after tests that call the factory successfully —
+ otherwise the agent CLI cutoff (armed by ``silence_logging_for_agent_cli`` inside
+ the factory) leaks into other tests in the suite.
+ """
original_mode = PrettyPrinter.mode
root_logger = logging.getLogger()
original_level: int = root_logger.level
+ original_disable = logging.root.manager.disable
yield
PrettyPrinter.mode = original_mode
root_logger.setLevel(original_level)
+ logging.disable(original_disable)
def test_successful_initialization(self, mocker: MockerFixture) -> None:
"""Should return the Pipelex instance when make() succeeds."""
diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
index 233766103..85992d3a6 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_init_overrides.py
@@ -6,9 +6,10 @@ Regression guard: a user TOML with ``[pipelex.log_config.package_log_levels] pip
``log.warning``) onto stderr — corrupting the JSON error envelope that downstream agent
consumers parse. The bulletproof cutoff is ``silence_logging_for_agent_cli`` which is
called at the very start of every agent CLI entry point and uses
-``logging.disable(LOGGING_LEVEL_OFF)`` — a process-global threshold that blocks any
-record for any logger, regardless of which package emits or what level it's configured
-at. The ``config_overrides`` carry defense-in-depth pins on top.
+``logging.disable(sys.maxsize)`` — a process-global threshold that blocks any record
+for any logger at any level (including custom levels above CRITICAL), regardless of
+which package emits or what level it's configured at. The ``config_overrides`` carry
+defense-in-depth pins on top.
The agent CLI has NO ``--log-level`` flag. Suppression is unconditional by design — the
CLI is machine-consumed and stderr is reserved for the structured error envelope.
@@ -17,6 +18,7 @@ CLI is machine-consumed and stderr is reserved for the structured error envelope
from __future__ import annotations
import logging
+import sys
from typing import TYPE_CHECKING
import pytest
@@ -29,7 +31,7 @@ from pipelex.cli.agent_cli.commands.agent_cli_factory import (
make_pipelex_for_agent_cli,
)
from pipelex.system.console_target import ConsoleTarget
-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF, LogLevel
+from pipelex.tools.log.log_levels import LogLevel
class TestAgentCliFactoryInitOverrides:
@@ -63,9 +65,9 @@ class TestAgentCliFactoryInitOverrides:
table / pretty-print fires.
2. ``silence_logging_for_agent_cli`` (called before ``Pipelex.make`` so the
cutoff is active during init) must have armed ``logging.disable`` at
- ``LOGGING_LEVEL_OFF`` — the bulletproof handler-side cutoff that blocks
- every record for every logger, including third-party libraries we don't
- enumerate.
+ ``sys.maxsize`` — the bulletproof handler-side cutoff that blocks every
+ record for every logger at every level (including custom levels above
+ CRITICAL), including third-party libraries we don't enumerate.
"""
mock_pipelex = mocker.MagicMock()
mock_make = mocker.patch(
@@ -82,8 +84,8 @@ class TestAgentCliFactoryInitOverrides:
assert log_config_overrides["console_log_target"] is ConsoleTarget.STDERR
assert log_config_overrides["console_print_target"] is ConsoleTarget.STDERR
- assert logging.root.manager.disable == LOGGING_LEVEL_OFF, (
+ assert logging.root.manager.disable == sys.maxsize, (
"make_pipelex_for_agent_cli must call silence_logging_for_agent_cli (which arms "
- "logging.disable at LOGGING_LEVEL_OFF) BEFORE Pipelex.make — otherwise any "
+ "logging.disable at sys.maxsize) BEFORE Pipelex.make — otherwise any "
"third-party logger configured at INFO/WARNING leaks records during init."
)
diff --git a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
index bd89913fd..620881773 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_factory_suppression.py
@@ -13,21 +13,36 @@ if TYPE_CHECKING:
from pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_cli
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
+# Loggers armed by the test below at non-default levels. Their ``.level`` attribute is
+# snapshot in the autouse fixture and restored after each test so the per-logger state
+# does not leak into the rest of the suite (other tests may rely on the default NOTSET).
+_ARMED_LOGGER_NAMES: tuple[str, ...] = (
+ "pipelex",
+ "anthropic",
+ "httpx",
+ "some.unknown.transitive.dep",
+)
+
class TestAgentCliFactorySuppression:
"""After successful init, the factory should silence pretty-print and Python logging."""
@pytest.fixture(autouse=True)
def _restore_globals(self):
- """Save and restore PrettyPrinter.mode and the process-global logging disable
- threshold around each test so the agent CLI cutoff does not leak into the rest
- of the suite.
+ """Save and restore PrettyPrinter.mode, the process-global logging disable
+ threshold, AND the per-logger ``.level`` for every logger this test class arms.
+ Without restoring the per-logger levels, the ``setLevel`` calls in the test body
+ leak into shared module-level loggers (``pipelex``, ``anthropic``, ``httpx``, ...)
+ and affect any other test that assumes default NOTSET.
"""
original_mode = PrettyPrinter.mode
original_disable = logging.root.manager.disable
+ original_levels = {name: logging.getLogger(name).level for name in _ARMED_LOGGER_NAMES}
yield
PrettyPrinter.mode = original_mode
logging.disable(original_disable)
+ for name, level in original_levels.items():
+ logging.getLogger(name).setLevel(level)
def test_sets_silent_pretty_print_mode(self, mocker: MockerFixture) -> None:
"""PrettyPrinter.mode should be SILENT after make_pipelex_for_agent_cli succeeds."""
diff --git a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
index 64af8210d..b7ce52060 100644
--- a/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
+++ b/tests/unit/pipelex/cli/test_agent_cli_output_discipline.py
@@ -18,6 +18,7 @@ would ship green. This module covers the real side effects.
from __future__ import annotations
import logging
+import sys
from typing import TYPE_CHECKING
import pytest
@@ -30,28 +31,42 @@ from pipelex.cli.agent_cli.commands.agent_cli_factory import (
silence_logging_for_agent_cli,
)
from pipelex.system.console_target import ConsoleTarget
-from pipelex.tools.log.log_levels import LOGGING_LEVEL_OFF
from pipelex.tools.misc.pretty import PrettyPrinter, PrettyPrintMode
+# Loggers armed by the test below at non-OFF levels. Their ``.level`` attribute is
+# snapshot in the autouse fixture and restored after each test so the per-logger state
+# does not leak into the rest of the suite (other tests may rely on the default NOTSET).
+_ARMED_LOGGER_NAMES: tuple[str, ...] = (
+ "anthropic",
+ "httpx",
+ "some.transitive.dep.we.never.heard.of",
+)
+
class TestAgentCliOutputDiscipline:
@pytest.fixture(autouse=True)
def _restore_globals(self):
- """Restore PrettyPrinter.mode and process-global ``logging.disable`` after each
- test so the agent CLI cutoff does not leak into other tests in the suite.
+ """Restore PrettyPrinter.mode, the process-global ``logging.disable`` threshold,
+ AND the per-logger ``.level`` for every logger this test class arms — otherwise
+ the levels set on shared module-level loggers (``anthropic``, ``httpx``, ...) leak
+ into other tests that assume default NOTSET.
"""
original_mode = PrettyPrinter.mode
original_disable = logging.root.manager.disable
+ original_levels = {name: logging.getLogger(name).level for name in _ARMED_LOGGER_NAMES}
yield
PrettyPrinter.mode = original_mode
logging.disable(original_disable)
+ for name, level in original_levels.items():
+ logging.getLogger(name).setLevel(level)
def test_silence_logging_disables_every_third_party_logger_regardless_of_configured_level(self) -> None:
"""Regression: ``silence_logging_for_agent_cli`` must call ``logging.disable`` at
- ``LOGGING_LEVEL_OFF`` so that every logger — pipelex, anthropic, httpx, botocore,
+ ``sys.maxsize`` so that every logger — pipelex, anthropic, httpx, botocore,
openai, anything a transitive dep ever creates — has ``isEnabledFor`` short-circuit
- to False before its own level check fires. This is the only defense that scales
- without an enumeration of package names.
+ to False before its own level check fires, at every level (including custom levels
+ above CRITICAL). This is the only defense that scales without an enumeration of
+ package names.
"""
# Arm a couple of loggers at INFO/WARNING as if a downstream library had configured
# them. The global disable must override their own level checks.
@@ -65,7 +80,7 @@ class TestAgentCliOutputDiscipline:
silence_logging_for_agent_cli()
- assert logging.root.manager.disable == LOGGING_LEVEL_OFF
+ assert logging.root.manager.disable == sys.maxsize
assert not anthropic_logger.isEnabledFor(logging.INFO)
assert not anthropic_logger.isEnabledFor(logging.CRITICAL)
assert not httpx_logger.isEnabledFor(logging.WARNING)
diff --git a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
index f0fcef57d..a0ee65d61 100644
--- a/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
+++ b/tests/unit/pipelex/cli/test_agent_doctor_cmd.py
@@ -29,7 +29,7 @@ class TestAgentDoctorCmd:
``setup_doctor_runtime`` instantiates a PipelexHub, loads config from disk, and
calls ``log.configure`` (once-per-process). ``apply_agent_cli_output_discipline``
mutates global PrettyPrinter mode and the hub's console target.
- ``silence_logging_for_agent_cli`` arms ``logging.disable`` at LOGGING_LEVEL_OFF
+ ``silence_logging_for_agent_cli`` arms ``logging.disable`` at ``sys.maxsize``
— process-global and would leak into other tests in the suite. All three are
out of scope for these tests — they cover the command's output shape, not the
runtime bootstrap mechanics.