Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

30.3 KB
Raw Blame
"""Nothing in ``src/`` may log or serialise a whole LLM object.

Routing the Ollama bearer token through ``client_kwargs`` is the only way
it reaches the httpx clients (``ChatOllama`` has no ``headers`` field),
but ``client_kwargs`` is a plain pydantic field: langchain_ollama marks
it neither ``exclude`` nor ``repr=False``, so the token is present in
``repr(llm)``, ``llm.model_dump()`` and ``dumpd(llm)`` — the
``serialized`` payload every LangChain callback receives. There is no
supported way to hide it without subclassing a third-party model, so the
exposure is contained by contract instead.

Today the only consumer is ``metrics.token_counter.TokenCountingCallback
.on_llm_start``, which reads named scalar keys (``_type``, ``name``,
``kwargs["model"]``) and neither logs nor persists the dict.

What this contract actually detects
-----------------------------------
It is a syntactic (``ast``) rule over the shipped package, so it sees
*shapes*, not values. It reports, anywhere in a module:

1. a whole LLM object or the whole ``serialized`` dict reaching a logging
   call — every ``logger.<level>(...)`` plus ``logger.bind(...)``, in a
   positional argument, a keyword value, or an f-string placeholder;
2. the same values reaching a serialisation call — ``str``/``repr``/
   ``print``/``format``/``pprint``/``pformat``, ``dumpd``/``dumps``,
   ``json.dumps``/``yaml.dump``/``pickle.dumps``, and the
   ``.model_dump()``/``.model_dump_json()``/``.dict()``/``.json()``/
   ``.to_dict()`` methods on the object itself — whether or not the
   result is then logged, so ``row.payload = llm.model_dump()`` counts;
3. ``<llm>.__dict__`` and the credential-bearing ``*client_kwargs``
   attributes in any of those positions — including a read *into* one:
   ``llm.client_kwargs["headers"]``, the chained
   ``…["headers"]["Authorization"]`` and ``llm.client_kwargs.get(…)`` all
   count, because that mapping *is* the bearer token;
4. a whole LLM object interpolated into *any* f-string, logged or not,
   because the resulting string is one assignment away from a sink;
5. the ``serialized`` dict passed to any call at all, bare or wrapped in
   a container (``audit([serialized])``, ``store(**{"p": serialized})``)
   — the callback may read named keys off it (``serialized["_type"]``,
   ``serialized.get("name")``), never hand the dict on;
6. any of the above inside a comprehension, in either the produced value
   or the iterable it is drawn from (``[m for m in [llm]]``).

An "LLM object" here means a bare name in ``_LLM_NAMES`` or an attribute
read whose final name is in ``_LLM_ATTRS`` (``self.llm``, ``self._llm``,
``agent.chat_model``, …) — and reading one LLM-valued attribute *off*
another (``self.llm.base_llm``, the wrapper unwrap that hands a caller
the raw model) is itself such a read.

What it does not detect
-----------------------
- **A scalar field of an LLM already identified.** ``self.base_llm.model``
  and its private spellings (``_model``, ``llm_model``, ``_chat_model``)
  are langchain's model-*name* string, so the chain stops there. Any other
  attribute name is either credential-bearing, another handle on the
  object, or outside the lists entirely.
- **Renamed or derived values.** ``x = llm`` then ``logger.info(x)``,
  ``payload = llm.model_dump()`` logged three functions later, a dict
  literal built field-by-field from the object, or an LLM stored in a
  container and read back (``llms[0]``, ``self._pool["chat"]``). There is
  no dataflow analysis here. The same limit covers a credential mapping
  reached through a call result (``getattr(llm, "client_kwargs")["headers"]``,
  ``vars(llm)["client_kwargs"]``), its ``.values()``/``.items()``/``.copy()``
  views, and a bare local that holds the mapping before it is attached to
  the model (``client_kwargs = {...}`` in the provider factory).
- **Third-party sinks.** Anything langchain, an SDK, a callback handler
  from another package, or a tracing integration does with the object
  after it leaves ``src/`` — including the ``serialized`` payload
  langchain itself constructs and dispatches to every registered handler.
- **A class's own ``__str__``/``__repr__``/``__format__`` restating the
  object it wraps** — ``repr``/``str``/``format`` of a whole LLM, or one
  in the f-string the method returns. That does not create a sink, it
  restates the exposure the contract already assumes: it does mean
  ``repr(RateLimitedLLMWrapper(...))`` carries the token exactly as
  ``repr(llm)`` does. *Only* that forwarding is exempt — a logger call, a
  ``model_dump``/``dumpd``/``json.dumps``/``yaml.dump``, a persistence
  call, a ``__dict__`` read and a ``serialized`` hand-off are all still
  reported inside such a method, as is anything in a function nested in
  it. The method must be defined directly in a ``class`` body; a
  module-level ``def __repr__`` is an ordinary function.
- **Code outside the shipped package**: tests, scripts and notebooks.
- **Names outside the two lists below.** The sets are literal; an LLM
  held in a variable called something else is invisible to this rule.

So this is a tripwire against the idioms that would leak the token
tomorrow, not a proof that it cannot leak. The underlying exposure is
unfixed and unfixable from here without subclassing a third-party model.

``getattr(llm, "base_url", ...)`` and ``llm.some_attr`` stay allowed —
naming one attribute is not dumping the object, unless that attribute is
a credential mapping or another model — as do the identity and type
predicates ``type(llm)``, ``id(llm)``, ``isinstance(llm, X)`` and
``llm is None``, which reveal nothing.
"""

import ast
from pathlib import Path

import pytest

import local_deep_research

SRC = Path(local_deep_research.__file__).parent

# --- BEGIN PREDICATE ---
_LOG_METHODS = {
    "debug",
    "info",
    "warning",
    "warn",
    "error",
    "exception",
    "critical",
    "success",
    "trace",
    "log",
    # loguru's contextual binder: `logger.bind(llm=llm).info(...)` puts the
    # object in the record's `extra` dict, which every sink formats.
    "bind",
}
# Called as plain functions: `repr(llm)`, `dumpd(llm)`. `json.dumps(x)`
# reaches this rule through _DUMP_METHODS instead.
_DUMP_FUNCS = {
    "str",
    "repr",
    "format",
    "print",
    "pprint",
    "pformat",
    "dumpd",
    "dumps",
}
# Called as methods. Two distinct positions matter: the *receiver*
# (`llm.model_dump()`) and the *arguments* (`json.dumps(serialized)`,
# `yaml.dump(llm)`).
_DUMP_METHODS = {
    "dump",
    "dumps",
    "dumpd",
    "model_dump",
    "model_dump_json",
    "dict",
    "json",
    "to_dict",
    "to_json",
    "serialize",
}
# Bare variable names that hold a whole LLM.
_LLM_NAMES = {
    "llm",
    "_llm",
    "base_llm",
    "raw_llm",
    "chat_model",
    "chat_llm",
    "wrapped_llm",
    "model_instance",
    "llm_instance",
    "provider_llm",
}
# Attribute names that hold langchain's model-*name* string rather than a
# model object: `ChatOllama.model`, `ChatOpenAI.model`, and the private
# spellings this tree uses for the same value. Read off an LLM this rule
# has already identified (`self.base_llm.model`), one of these is a single
# scalar field and the chain stops there; read off anything else
# (`strategy.model`) the owner is unknown and the value is taken to be the
# object.
_SCALAR_LLM_FIELDS = {
    "model",
    "_model",
    "llm_model",
    "_chat_model",
}
# Attribute names that hold a whole LLM: `self.llm`, `handler._llm`,
# `strategy.model`. Checked on the *final* name of the attribute chain,
# so the owner can be anything.
_LLM_ATTRS = _LLM_NAMES | _SCALAR_LLM_FIELDS
# The callback payload. Named keys may be read off it; the dict itself
# may not be logged, serialised, or handed to another call.
_SERIALIZED = "serialized"
_CREDENTIAL_ATTRS = {
    "client_kwargs",
    "sync_client_kwargs",
    "async_client_kwargs",
}
# Mapping reads that are equivalent to a subscript: `d.get("k")` and
# `d["k"]` return the same value, so both are followed to their base.
_MAPPING_READS = {"get", "setdefault", "pop"}
# Reveal nothing about the object's contents, so an LLM reaching one of
# these is not a leak — and neither is whatever the call returns.
_SAFE_FUNCS = {
    "type",
    "id",
    "isinstance",
    "issubclass",
    "callable",
    "hasattr",
    "len",
    "bool",
    "getattr",
}


def _llm_label(node):
    """Return a printable name if ``node`` is a whole LLM / the payload."""
    if isinstance(node, ast.Name):
        if node.id in _LLM_NAMES:
            return node.id
        if node.id == _SERIALIZED:
            return _SERIALIZED
        return None
    if isinstance(node, ast.Attribute) and node.attr in _LLM_ATTRS:
        # `self.base_llm.model` is langchain's model-*name* string, a
        # scalar field of an LLM we have already identified — not a
        # second LLM — so that one chain stops here. Every other name in
        # the set holds an object, and reading it off an LLM is exactly
        # the wrapper unwrap (`self.llm.base_llm`) that hands a caller
        # the credential-bearing model: those keep the label.
        if (
            node.attr in _SCALAR_LLM_FIELDS
            and _llm_label(node.value) is not None
        ):
            return None
        return f".{node.attr}"
    return None


def _is_safe_call(node):
    """``type(llm)``/``isinstance(llm, X)``/``getattr(llm, "base_url")``.

    ``getattr`` is exempt only when the attribute it names is not itself
    credential-bearing.
    """
    if not (isinstance(node, ast.Call) and isinstance(node.func, ast.Name)):
        return False
    if node.func.id not in _SAFE_FUNCS:
        return False
    if node.func.id == "getattr" and len(node.args) >= 2:
        wanted = node.args[1]
        if (
            isinstance(wanted, ast.Constant)
            and wanted.value in _CREDENTIAL_ATTRS
        ):
            return False
    return True


def _is_identity_compare(node):
    """``llm is None`` / ``llm in registry`` — a boolean, not a dump."""
    return isinstance(node, ast.Compare) and all(
        isinstance(op, (ast.Is, ast.IsNot, ast.In, ast.NotIn))
        for op in node.ops
    )


def _indexed_base(node):
    """Follow a ``[...]``/``.get(...)`` chain down to what is indexed.

    ``llm.client_kwargs["headers"]`` *is* ``{"Authorization": "Bearer …"}``
    — indexing a credential mapping is not reading one named scalar off
    the object, so the chain is followed to its base and reported.
    ``serialized["_type"]`` and ``serialized.get("name")`` bottom out in
    the callback payload, which the contract explicitly allows to be read
    by key, so that base returns nothing.
    """
    depth = 0
    while True:
        if isinstance(node, ast.Subscript):
            node, depth = node.value, depth + 1
            continue
        if (
            isinstance(node, ast.Call)
            and isinstance(node.func, ast.Attribute)
            and node.func.attr in _MAPPING_READS
        ):
            node, depth = node.func.value, depth + 1
            continue
        break
    if depth == 0:
        return None
    if isinstance(node, ast.Attribute) and node.attr in _CREDENTIAL_ATTRS:
        owner = _llm_label(node.value)
        return None if owner is None else f"{owner}.{node.attr}"
    label = _llm_label(node)
    return None if label in (None, _SERIALIZED) else label


def _payload_leaks(node):
    """Whole-object leaks reachable from a value that *is* being emitted.

    Descent stops at a named scalar attribute read (``llm.model_name`` is
    one string, not the object) and at a subscript of anything but a
    credential mapping or an LLM (``serialized["_type"]``).
    """
    if node is None or _is_safe_call(node) or _is_identity_compare(node):
        return []
    label = _llm_label(node)
    if label is not None:
        return [(node.lineno, f"`{label}`")]
    if isinstance(node, ast.Attribute):
        owner = _llm_label(node.value)
        if owner is not None and (
            node.attr in _CREDENTIAL_ATTRS or node.attr == "__dict__"
        ):
            return [(node.lineno, f"`{owner}.{node.attr}`")]
        # A named attribute of an LLM is allowed; keep looking inside
        # anything that is not itself a plain name/attribute chain.
        if isinstance(node.value, (ast.Name, ast.Attribute)):
            return []
        return _payload_leaks(node.value)
    if isinstance(node, ast.Subscript):
        base = _indexed_base(node)
        if base is not None:
            return [(node.lineno, f"`{base}` is read by key")]
        return []
    if isinstance(node, ast.IfExp):
        # `'yes' if llm else 'no'` — the test is a truthiness guard in the
        # same class as `llm is None`; only the branches are emitted.
        return _payload_leaks(node.body) + _payload_leaks(node.orelse)
    if isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
        return []
    if isinstance(node, ast.Call):
        base = _indexed_base(node)
        if base is not None:
            return [(node.lineno, f"`{base}` is read by key")]
        out = []
        for arg in node.args:
            out += _payload_leaks(arg)
        for kw in node.keywords:
            out += _payload_leaks(kw.value)
        out += _payload_leaks(node.func)
        return out
    if isinstance(node, ast.comprehension):
        # The loop target is a fresh binding, not a read of the object;
        # the iterable and the guards are ordinary reads.
        out = _payload_leaks(node.iter)
        for cond in node.ifs:
            out += _payload_leaks(cond)
        return out
    # No `ast.expr` filter: `ast.comprehension` is not an expression, and
    # skipping it left `[m for m in [llm]]` unvisited below its `elt`.
    out = []
    for child in ast.iter_child_nodes(node):
        out += _payload_leaks(child)
    return out


def _serialized_reads(node):
    """Bare ``serialized`` reads in a value handed to a call.

    Wrapping it changes nothing: ``audit([serialized])``,
    ``audit({"p": serialized})`` and ``store(**{"p": serialized})`` hand
    the dict on exactly as ``audit(serialized)`` does. A read *by key*
    inside the container (``{serialized["_type"]: 1}``) is still allowed.
    """
    if isinstance(node, ast.Name):
        return [node] if node.id == _SERIALIZED else []
    if isinstance(node, (ast.List, ast.Tuple, ast.Set, ast.Dict)):
        out = []
        for child in ast.iter_child_nodes(node):
            out += _serialized_reads(child)
        return out
    return []


def _call_leaks(node):
    """Node-local rules for a single ``ast.Call``."""
    func = node.func
    out = []
    args = list(node.args) + [kw.value for kw in node.keywords]

    is_log = isinstance(func, ast.Attribute) and func.attr in _LOG_METHODS
    if is_log:
        for arg in args:
            out += [
                (line, f"{what} reaches `.{func.attr}(...)`")
                for line, what in _payload_leaks(arg)
            ]

    is_dump_func = isinstance(func, ast.Name) and func.id in _DUMP_FUNCS
    is_dump_method = (
        isinstance(func, ast.Attribute) and func.attr in _DUMP_METHODS
    )
    if is_dump_func or is_dump_method:
        sink = func.id if is_dump_func else f".{func.attr}"
        for arg in args:
            out += [
                (line, f"{what} is serialised by `{sink}(...)`")
                for line, what in _payload_leaks(arg)
            ]
    if is_dump_method:
        # The receiver: `llm.model_dump()`, `self.llm.dict()`.
        receiver = _llm_label(func.value)
        if receiver is not None:
            out.append(
                (
                    node.lineno,
                    f"`{receiver}` is serialised by `.{func.attr}()`",
                )
            )

    # The callback payload may be read by key, never handed on.
    if not (is_log or _is_safe_call(node)):
        for arg in args:
            for name in _serialized_reads(arg):
                out.append(
                    (
                        name.lineno,
                        f"`{_SERIALIZED}` is passed whole to another call",
                    )
                )
    return out


# A wrapper's own ``__repr__`` forwarding to the object it wraps does not
# create a sink: it makes ``repr(wrapper)`` exactly as revealing as
# ``repr(llm)``, which is the exposure this contract already assumes and
# guards. ``RateLimitedLLMWrapper.__str__``/``__repr__``
# (``web_search_engines/rate_limiting/llm/wrapper.py``) is the site this
# carve-out exists for, but these names are common — 62 of them across 22
# files, mostly SQLAlchemy models — so the exemption is kept as narrow as
# the justification: it covers only the forwarding stringify itself
# (``repr``/``str``/``format`` of an LLM, or an LLM in the f-string the
# method returns), only in a method defined directly in a ``ClassDef``
# body, and not in any function nested inside it. A logger call, a
# ``model_dump``, a persistence call or a ``serialized`` hand-off is a
# real sink wherever it is written.
_REPR_DUNDERS = {"__str__", "__repr__", "__format__"}
# Restating the wrapped object's own text is what these methods are for.
_FORWARDING_REPR_FUNCS = {"str", "repr", "format"}


def _own_scope_nodes(func):
    """Every node of ``func`` that is not inside a nested scope."""
    inside = set()
    pending = list(ast.iter_child_nodes(func))
    while pending:
        node = pending.pop()
        inside.add(id(node))
        if isinstance(
            node,
            (
                ast.FunctionDef,
                ast.AsyncFunctionDef,
                ast.Lambda,
                ast.ClassDef,
            ),
        ):
            # A nested `def helper(): print(llm)` is ordinary code that
            # happens to sit in a dunder; it gets no carve-out.
            continue
        pending += list(ast.iter_child_nodes(node))
    return inside


def _repr_dunder_nodes(tree):
    """Nodes in a class's own ``__str__``/``__repr__``/``__format__``.

    The method must be a direct child of a ``ClassDef`` body: a
    module-level ``def __repr__`` is an ordinary function, and naming one
    of these must not switch the contract off for a whole module.
    """
    inside = set()
    for owner in ast.walk(tree):
        if not isinstance(owner, ast.ClassDef):
            continue
        for node in owner.body:
            if (
                isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))
                and node.name in _REPR_DUNDERS
            ):
                inside |= _own_scope_nodes(node)
    return inside


def _is_forwarding_repr(node):
    """``repr(self.base_llm)`` — the wrapped object restating itself.

    Only a whole LLM qualifies: ``repr(llm.client_kwargs)`` is a
    credential read wearing a repr, and ``str(serialized)`` is the
    callback payload. Neither is what a wrapper's dunder is for.
    """
    if not (
        isinstance(node.func, ast.Name)
        and node.func.id in _FORWARDING_REPR_FUNCS
    ):
        return False
    return bool(node.args) and _is_forwarded_value(node.args[0])


def _is_forwarded_value(node):
    """A whole LLM, or its repr — all a wrapper's dunder may restate."""
    if isinstance(node, ast.Call):
        return _is_forwarding_repr(node)
    label = _llm_label(node)
    return label is not None and label != _SERIALIZED


def _violations(source: str, label: str):
    """Yield ``"<label>:<line> <reason>"`` for every offending sink."""
    tree = ast.parse(source, filename=label)
    exempt = _repr_dunder_nodes(tree)
    found = []
    for node in ast.walk(tree):
        in_repr = id(node) in exempt
        if isinstance(node, ast.Call):
            # Inside a repr dunder the wrapped object's own text may be
            # restated — and nothing else. Every other sink stays live.
            if in_repr and _is_forwarding_repr(node):
                continue
            found += _call_leaks(node)
        elif isinstance(node, ast.JoinedStr):
            # An f-string is one assignment away from any sink, so the
            # interpolation itself is the violation — except where such a
            # method restates the wrapped object, which is the f-string
            # `f"RateLimitedLLMWrapper({repr(self.base_llm)})"` returns.
            for part in node.values:
                if not isinstance(part, ast.FormattedValue):
                    continue
                if in_repr and _is_forwarded_value(part.value):
                    continue
                found += [
                    (line, f"{what} is interpolated into an f-string")
                    for line, what in _payload_leaks(part.value)
                ]
        elif isinstance(node, ast.Attribute) and node.attr == "__dict__":
            owner = _llm_label(node.value)
            if owner is not None:
                found.append((node.lineno, f"`{owner}.__dict__` is read"))
    seen = set()
    out = []
    for line, reason in found:
        key = (line, reason)
        if key in seen:
            continue
        seen.add(key)
        out.append(f"{label}:{line} {reason}")
    return sorted(out)


# --- END PREDICATE ---


class TestTheGuardActuallyDetects:
    """A contract test that cannot fail is worthless; prove it fires."""

    @pytest.mark.parametrize(
        "snippet",
        [
            'logger.info(f"built {llm}")',
            "logger.debug(llm)",
            'logger.info("cfg %s", llm.client_kwargs)',
            "json.dumps(serialized)",
            "logger.warning(repr(llm))",
            'logger.info(f"start {serialized}")',
            # Shapes the first round of this contract missed.
            'logger.info(f"{repr(llm)}")',
            'logger.info(f"{llm.model_dump()}")',
            'logger.info("%s", llm.model_dump())',
            "row.payload = llm.model_dump()",
            "store(dumpd(llm))",
            "session.add(Row(payload=serialized))",
            'logger.info(f"{self.llm}")',
            'logger.bind(llm=llm).info("made")',
            'msg = f"llm={llm}"',
            "print(llm)",
            # Neighbours of the above.
            'logger.info(f"{self._llm}")',
            'logger.debug(f"{self.model}")',
            "yaml.dump(llm)",
            "pickle.dumps(self.chat_model)",
            "payload = self.llm.__dict__",
            'logger.info("cfg %s", self.llm.sync_client_kwargs)',
            'logger.info(getattr(llm, "client_kwargs", None))',
            "audit(serialized)",
            "logger.bind(payload=serialized).debug('x')",
            "return llm.model_dump_json()",
            "blob = self.base_llm.to_dict()",
            # Indexing a credential mapping reads the token itself.
            'logger.error("cfg %s", llm.client_kwargs["headers"])',
            "logger.error(f\"{llm.client_kwargs['headers']}\")",
            'logger.info(llm.client_kwargs["headers"]["Authorization"])',
            "logger.info(f\"{llm.sync_client_kwargs['headers']}\")",
            "logger.info(f\"{llm.client_kwargs.get('headers')}\")",
            'logger.info(f"{llm.async_client_kwargs[0]}")',
            # Unwrapping a wrapper hands over the wrapped model.
            'logger.info(f"{self.llm.base_llm}")',
            'logger.info(f"{llm.base_llm}")',
            'logger.info(f"{self.model.base_llm}")',
            'logger.info(f"{self.llm.base_llm.client_kwargs}")',
            "self.llm.base_llm.model_dump()",
            "repr(self.llm.base_llm)",
            # A comprehension's iterable is read like any other value.
            'logger.info(f"{[m for m in [llm]]}")',
            'logger.info(", ".join(str(m) for m in [llm]))',
            # Wrapping the payload does not stop it being handed on.
            "audit([serialized])",
            'audit({"p": serialized})',
            'store(**{"p": serialized})',
            # The repr-dunder exemption covers the forwarded repr and
            # nothing else: not a logger, not a dump, not persistence,
            # not the payload, not a nested `def`, not a bare function.
            "class W:\n"
            "    def __repr__(self):\n"
            "        logger.info(self.base_llm)\n"
            "        return 'W'",
            "class W:\n"
            "    def __repr__(self):\n"
            "        session.add(Row(p=self.llm.model_dump()))\n"
            "        return 'x'",
            "class W:\n"
            "    def __repr__(self):\n"
            "        audit(serialized)\n"
            "        return 'x'",
            "class W:\n"
            "    def __format__(self, spec):\n"
            "        store(dumpd(llm))\n"
            "        return 'x'",
            "class W:\n"
            "    def __repr__(self):\n"
            "        def helper():\n"
            "            return repr(llm)\n"
            "        return 'x'",
            "class NotAnLLM:\n"
            "    def __repr__(self):\n"
            "        yaml.dump(llm)\n"
            "        return 'x'",
            "def __repr__(self):\n    return repr(self.llm)",
            "class W:\n"
            "    def __repr__(self):\n"
            "        return repr(llm.client_kwargs)",
        ],
        ids=(
            "fstring-llm",
            "bare-llm",
            "client-kwargs-attr",
            "json-dumps-serialized",
            "repr-llm",
            "fstring-serialized",
            "nested-repr-in-fstring",
            "model-dump-in-fstring",
            "model-dump-printf-arg",
            "model-dump-persisted",
            "dumpd-into-a-store",
            "serialized-persisted",
            "self-llm-in-fstring",
            "logger-bind-kwarg",
            "fstring-outside-a-call",
            "print-llm",
            "self-underscore-llm",
            "self-model-attr",
            "yaml-dump",
            "pickle-dumps-attr",
            "dunder-dict",
            "credential-attr-on-self",
            "getattr-credential-attr",
            "serialized-to-any-call",
            "bind-serialized",
            "model-dump-json-returned",
            "to-dict-on-attr",
            "credential-subscript",
            "credential-subscript-in-an-fstring",
            "credential-subscript-chained",
            "sync-credential-subscript",
            "credential-get",
            "async-credential-index",
            "wrapper-unwrap-off-self-llm",
            "wrapper-unwrap-off-a-bare-llm",
            "wrapper-unwrap-off-self-model",
            "credential-behind-an-unwrap",
            "model-dump-behind-an-unwrap",
            "repr-of-an-unwrap",
            "comprehension-iterable",
            "generator-iterable",
            "serialized-in-a-list",
            "serialized-in-a-dict",
            "serialized-in-a-splat",
            "logger-inside-a-repr-dunder",
            "persistence-inside-a-repr-dunder",
            "serialized-inside-a-repr-dunder",
            "dumpd-inside-a-format-dunder",
            "nested-def-inside-a-repr-dunder",
            "repr-dunder-of-an-unrelated-class",
            "module-level-repr-dunder",
            "credential-repr-inside-a-repr-dunder",
        ),
    )
    def test_offending_shapes_are_reported(self, snippet):
        assert _violations(snippet, "<snippet>")

    @pytest.mark.parametrize(
        "snippet",
        [
            "logger.info(f\"url {getattr(llm, 'base_url', None)}\")",
            'logger.debug(f"model {llm.model_name}")',
            'model_name = serialized["_type"]',
            'logger.info("count %d", len(prompts))',
            # Identity and type predicates reveal nothing (R2.4).
            'logger.info(f"{type(llm).__name__}")',
            'logger.info(f"{id(llm)}")',
            'logger.info(f"{llm is None}")',
            'logger.info("%s", isinstance(llm, ChatOpenAI))',
            "logger.debug(f\"{serialized.get('name')}\")",
            'logger.info("%s", serialized["kwargs"]["model"])',
            'logger.debug(f"{llm.__class__.__name__}")',
            "if llm is not None:\n    logger.info('ready')",
            'json.dumps({"model": llm.model_name})',
            # Truthiness guards and a wrapper restating its own repr.
            "logger.debug(f\"llm={'yes' if self.llm else 'no'}\")",
            'logger.debug(f"{not self.llm}")',
            "model = str(self.base_llm.model)",
            'logger.info(f"{self.llm.model}")',
            'logger.debug(f"{self.base_llm._model}")',
            "class W:\n"
            "    def __repr__(self):\n"
            "        return f'W({repr(self.base_llm)})'",
            'class W:\n    def __repr__(self):\n        return f"W({self.base_llm!r})"',
            "class W:\n    def __str__(self):\n        return str(self.base_llm)",
            # A comprehension over something that is not an LLM, and the
            # payload still read by key inside a container.
            'logger.info(f"{[p for p in prompts]}")',
            'logger.info(f"{[m for m in pool if m is llm]}")',
            'audit({serialized["_type"]: 1})',
        ],
        ids=(
            "getattr-one-attribute",
            "one-named-attribute",
            "subscript-outside-a-log-call",
            "unrelated-log",
            "type-name",
            "id-of",
            "is-none",
            "isinstance",
            "serialized-get-named-key",
            "serialized-nested-key",
            "class-name",
            "identity-guard",
            "one-attribute-into-json",
            "truthiness-guard-in-a-conditional",
            "negated-truthiness-guard",
            "model-name-field-of-an-llm",
            "model-field-of-an-identified-llm",
            "private-model-field-behind-an-unwrap",
            "wrapper-repr-dunder",
            "wrapper-repr-dunder-conversion",
            "wrapper-str-dunder",
            "comprehension-without-an-llm",
            "identity-guard-in-a-comprehension",
            "serialized-key-inside-a-container",
        ),
    )
    def test_legitimate_shapes_are_not_reported(self, snippet):
        assert not _violations(snippet, "<snippet>")


class TestNoSinkInSource:
    def test_no_module_logs_or_serialises_a_whole_llm(self):
        """Grep-style contract over the shipped package."""
        # Given
        modules = sorted(SRC.rglob("*.py"))
        assert len(modules) > 100, "source tree not found"

        # When
        offenders = []
        for path in modules:
            offenders += _violations(
                path.read_text(encoding="utf-8"),
                str(path.relative_to(SRC)),
            )

        # Then
        assert offenders == [], (
            "An LLM object or a callback `serialized` payload reaches a log "
            "or serialisation sink. The Ollama bearer token lives in "
            "`client_kwargs`, so that sink would persist a credential:\n"
            + "\n".join(offenders)
        )