Skip to content

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

16.2 KB
Raw Blame
"""Edge classification of ``fastapi_app._is_private_ip`` — the warning-path copy.

The repo carries three private-IP classifiers. The two that matter here
differ in fallback behaviour — this is not a hardened/unhardened pair:

- ``local_deep_research.security.network_utils.is_private_ip`` — it first
  short-circuits ``return True`` on a small set
  of known localhost spellings (``localhost``, ``127.0.0.1``, ``[::1]``,
  ``0.0.0.0``), BEFORE any bracket handling; only then does
  it strip the brackets off a ``[...]``-wrapped literal and
  hand everything else straight to ``ipaddress``'s own ``is_private`` /
  ``is_loopback`` / ``is_link_local``, falling back to a ``.local``-suffix
  check when the string does not parse as an IP at all. It does no
  unwrapping of its own: IPv4-mapped, NAT64 (``64:ff9b::/96``) and 6to4
  (``2002::/16``) forms get whatever the running CPython's ``ipaddress``
  module says about the address as presented — and for the two tunnel
  prefixes, which ``ipaddress`` answers from its network table rather
  than from the embedded IPv4, that answer is not always right. A
  NAT64-wrapped RFC1918/link-local address (e.g. ``64:ff9b::c0a8:1``, or
  the metadata-address case) is classified public, while on a CPython
  build whose private-networks table lists ``2002::/16`` (older builds
  that lack that entry do not), every 6to4 address is classified private
  regardless of what it wraps. Both classes are covered by the deep classification
  suite in ``tests/security/test_middleware_and_proxy_trust_fastapi.py``,
  but not on the same footing:
  ``test_6to4_wrapping_a_private_address_counts_as_private`` is
  explicitly labelled CURRENT BEHAVIOUR, not desired behaviour, while
  ``test_nat64_wrapped_addresses_are_not_private`` is framed as
  load-bearing — its docstring says ``security/egress/policy.py`` relies
  on this classification (plus its own ``_is_nat64_wrapped_metadata``
  check) so a NAT64-wrapped cloud metadata address cannot pose as a local
  host. That test asserts ``64:ff9b::8.8.8.8`` and
  ``64:ff9b::169.254.169.254`` (the metadata address); the RFC1918
  example above, ``64:ff9b::c0a8:1``, is not itself asserted there — the
  public class is pinned via a link-local metadata address instead. For
  IPv4-mapped inputs this function takes whatever the running
  interpreter's ``ipaddress`` answers: from CPython 3.12.4 that follows
  the IPv4 address the mapped form wraps, while 3.12.0 to 3.12.3 report
  every mapped address private.
- ``web.fastapi_app._is_private_ip`` — a local copy that does the same kind
  of delegation, except that it first unwraps an IPv4-mapped address to
  the IPv4 address it wraps, and classifies every 6to4
  (``2002::/16``), Teredo (``2001::/32``) and NAT64 (``64:ff9b::/96``,
  ``64:ff9b:1::/48``) address as NOT private before consulting
  ``ipaddress`` (a transition address stands for a host beyond a relay
  or translator), and with different edges: no bracket
  stripping, no localhost-string shortcut, and "non-parseable (including
  empty string) counts as private" so TestClient peers behave like
  loopback — where the module above instead falls back to a
  ``.local``-suffix check and would *not* treat an empty string as
  private. It also omits the other copy's ``is_link_local`` clause, but
  that difference is textual only: on every CPython this project supports
  (``requires-python = ">=3.12,<3.15"``) ``169.254.0.0/16`` and
  ``fe80::/10`` are both entries in ``ipaddress``'s own private-networks
  table, so ``is_private`` already covers them and no input can tell the
  two clause lists apart. Before this file, no test imported *this* copy
  at all (a comment in ``tests/web/test_forwarded_proto_warning.py`` was
  its only reference).

The third classifier, ``NotificationURLValidator._is_private_ip``
(``security/notification_validator.py``), is a different animal: it
resolves a hostname and screens the resolved addresses as an SSRF gate
on the notification egress path, with its own knobs
(``allow_private_ips``, ``allow_nat64``, ``block_link_local``) and its
own suite. It is named here only so that "the two classifiers" above is
not read as a repo-wide count.

This copy is reach-restricted, which is why untested edges are a pinning
concern rather than a fix-me bug: it feeds exactly the two operator-warning
paths inside ``SecureCookieMiddleware`` —

- ``_maybe_warn_insecure_public`` (one-shot "serving HTTP to a public peer")
- ``_maybe_warn_untrusted_forwarded_proto`` (one-shot proxy-misconfig hint)

— and never the ``Secure``-flag decision itself: that decision is
``should_add_secure = not self.testing and is_https`` where ``is_https``
is ``scope.get("scheme") == "https"`` (in ``SecureCookieMiddleware``). It is
gated on the testing flag as well as the scheme, and the peer address is
not an input to it at all; it is pinned in
``tests/web/test_secure_cookie_middleware.py``. A misclassification here
therefore costs an operator a log line, not a cookie its ``Secure`` flag.

What this file pins:

1. The honest classification table (RFC 1918, loopback, IPv6 ULA/link-local,
   public v4/v6, empty, non-parseable).
2. IPv4-mapped peers (``::ffff:a.b.c.d``) are judged by the IPv4 address
   they wrap, unwrapped by this copy itself as
   ``network_utils.is_private_peer_ip`` does, so ``::ffff:8.8.8.8``
   classifies public on every interpreter. CPython 3.12.0 to 3.12.3
   answer a mapped address from their IPv6 private-networks table, where
   ``::ffff:0:0/96`` is listed private outright, and would classify every
   mapped address private; one test simulates that table answer so the
   unwrap is what its assertions observe.
3. NAT64 (``64:ff9b::/96``) peers never get mapped-address treatment
   from CPython: a NAT64-wrapped public IPv4 classifies public.
4. 6to4 (``2002::/16``), Teredo (``2001::/32``) and NAT64 local-use
   (``64:ff9b:1::/48``) peers classify NOT private, although CPython's
   network table may list them as private, depending on the build
   (Teredo through ``2001::/23``; 6to4 and NAT64 local-use only on
   builds whose table carries those entries, where a 6to4-wrapped
   PUBLIC IPv4 such as ``2002:808:808::1`` reads private; older builds
   that lack them do not). Classifying them private cost an operator the
   "serving HTTP to a public client" warning, and let a remote client
   arriving over a tunnel plant the forwarded-proto advice.

Deliberately NOT pinned: the ``or ip.is_loopback`` half of the classifier.
It is unreachable as a *decision* — ``127.0.0.0/8`` and ``::1/128`` are
literal entries in ``ipaddress``'s private-networks table on every
supported CPython (and IPv4-mapped loopback delegates to the embedded
IPv4, which is private), with no overlapping entry in the
private-networks EXCEPTIONS table — so ``is_private`` is already true
everywhere ``is_loopback`` is. Deleting the clause changes the verdict
for no input, which means no assertion can catch its removal and any
test claiming to do so would be pinning something else. The loopback rows
below are therefore honest coverage of loopback *inputs*, not a pin on
that clause.

Same disclosure for the ``if not ip_str: return True`` guard on the
empty string: ``ipaddress.ip_address("")`` raises ``ValueError``, which
the ``except ValueError: return True`` arm already answers True, so
deleting the guard changes no verdict either. ``_is_private_ip("")``
below is honest coverage of the empty *input*, not a pin on the guard.

If this copy ever grows a non-warning caller (rate-limit keying, proxy
trust, access control), these pins are the tripwire that forces the
IPv4-mapped question to be re-answered first.
"""

import ipaddress

import pytest

from local_deep_research.web.fastapi_app import _is_private_ip


class TestPrivateClassification:
    def test_rfc1918_blocks_are_private(self):
        assert _is_private_ip("10.0.0.1")
        assert _is_private_ip("172.16.0.1")
        assert _is_private_ip("192.168.1.1")

    def test_loopback_is_private(self):
        # Loopback INPUTS, not a pin on the ``or ip.is_loopback`` clause:
        # these stay True with that clause deleted, because 127.0.0.0/8
        # and ::1/128 are private-networks entries in their own right.
        # See the "Deliberately NOT pinned" note in the module docstring.
        assert _is_private_ip("127.0.0.1")
        assert _is_private_ip("127.255.255.254")
        assert _is_private_ip("::1")

    def test_ipv6_unique_local_and_link_local_are_private(self):
        assert _is_private_ip("fc00::1")
        assert _is_private_ip("fd12:3456:789a::1")
        assert _is_private_ip("fe80::1")

    def test_public_ipv4_is_not_private(self):
        assert not _is_private_ip("8.8.8.8")
        assert not _is_private_ip("1.1.1.1")

    def test_public_ipv6_is_not_private(self):
        assert not _is_private_ip("2001:4860:4860::8888")

    def test_empty_and_non_parseable_are_private(self):
        # "" and "testclient" (the Starlette TestClient peer) classify
        # private. The SUT docstring (``_is_private_ip``) gives the
        # reason as "Non-parseable strings are treated as private to
        # avoid adding Secure flag in test/dev contexts" — quoted here
        # exactly because that rationale is STALE: this classifier is not
        # an input to the Secure-flag decision at all (that decision is
        # `not self.testing and is_https` in SecureCookieMiddleware, as the
        # module docstring above records). What the fallback actually
        # buys today is quiet operator-warning paths. Tracked in #6263.
        #
        # The "" row is coverage of the empty INPUT, not a pin on the
        # `if not ip_str: return True` guard: ip_address("") raises
        # ValueError, so the except arm returns True anyway and deleting
        # the guard changes no verdict. Same disclosure shape as the
        # `or ip.is_loopback` note in the module docstring.
        assert _is_private_ip("")
        assert _is_private_ip("testclient")
        assert _is_private_ip("not-an-ip")


class TestMappedAddressSemantics:
    def test_ipv4_mapped_public_peer_classifies_public(self):
        # The SECURE direction: a mapped PUBLIC peer classifies public,
        # so it gets the "serving HTTP to a public client" warning and
        # cannot plant the forwarded-proto advice.
        assert not _is_private_ip("::ffff:8.8.8.8")

    def test_mapped_address_follows_the_wrapped_ipv4_on_any_interpreter(
        self, monkeypatch
    ):
        """Decided by unwrapping, not by the interpreter's table.

        A stock CPython 3.12.4 or later already delegates a mapped address
        to the IPv4 it wraps, so there deleting the unwrap changes no
        verdict. This simulates CPython 3.12.0 to 3.12.3, which answer
        from the IPv6 network table where ``::ffff:0:0/96`` is listed
        private outright, so the unwrap is what the assertions observe.
        """
        mapped_range = ipaddress.ip_network("::ffff:0:0/96")
        table_answer = ipaddress.IPv6Address.is_private

        def is_private_from_table(self):
            if self in mapped_range:
                return True
            return table_answer.fget(self)

        monkeypatch.setattr(
            ipaddress.IPv6Address,
            "is_private",
            property(is_private_from_table),
        )
        # The simulation is in effect: the stdlib now says "private".
        assert ipaddress.ip_address("::ffff:8.8.8.8").is_private is True

        assert _is_private_ip("::ffff:8.8.8.8") is False
        assert _is_private_ip("::ffff:10.0.0.1") is True
        assert _is_private_ip("::ffff:127.0.0.1") is True

    def test_ipv4_mapped_loopback_classifies_private(self):
        assert _is_private_ip("::ffff:127.0.0.1")

    def test_ipv4_mapped_rfc1918_peer_classifies_private(self):
        assert _is_private_ip("::ffff:192.168.1.10")

    def test_nat64_embedded_public_ipv4_classifies_public(self):
        # 64:ff9b::/96 (NAT64 well-known prefix) is NOT in CPython's IPv6
        # private-networks table, and — unlike ::ffff:0:0/96 — CPython
        # does not treat it as a wrapper either, so the embedded IPv4 is
        # never consulted. A NAT64-wrapped PUBLIC peer therefore
        # classifies public.
        #
        # This is NOT where the asymmetry with the mapped range lives:
        # _is_private_ip unwraps a mapped address to the IPv4 it wraps, so
        # a mapped public peer also classifies public
        # (test_ipv4_mapped_public_peer_classifies_public above). The
        # asymmetry is on the PRIVATE-wrapped side —
        # `::ffff:192.168.1.10` and `::ffff:127.0.0.1` classify private
        # via the embedded IPv4, while `64:ff9b::c0a8:1` (RFC1918
        # 192.168.0.1 wrapped for NAT64) classifies PUBLIC. That
        # NAT64-wrapped-private case is pinned as current behaviour in
        # tests/security/test_middleware_and_proxy_trust_fastapi.py
        # (test_nat64_wrapped_addresses_are_not_private), against the
        # other classifier; the module docstring above records it.
        assert not _is_private_ip("64:ff9b::808:808")

    @pytest.mark.parametrize(
        "peer",
        [
            # 6to4 wrapping a public IPv4 (8.8.8.8) and a private one
            # (10.0.0.1): neither is a peer on this host or LAN.
            "2002:808:808::1",
            "2002:a00:1::1",
            # Teredo: 2001::/32 sits inside the private 2001::/23 block.
            "2001:0:4136:e378:8000:63bf:3fff:fdd2",
            # RFC 8215 NAT64 local-use prefix, private in CPython's table
            # on builds that carry that entry.
            "64:ff9b:1::808:808",
            # Well-known NAT64 wrapping an RFC1918 address.
            "64:ff9b::a00:1",
            # A zone id does not change the classification.
            "2002:808:808::1%eth0",
        ],
    )
    def test_transition_addresses_classify_public(self, peer):
        # Each of these may read is_private=True in CPython, depending on
        # the build (the well-known NAT64 row is listed so the rule does
        # not depend on the network table). Classified private, a
        # 6to4/Teredo client silently got no "serving HTTP to a public client" warning and
        # could plant the forwarded-proto advice meant for a local
        # reverse proxy.
        assert not _is_private_ip(peer)

    def test_transition_rule_is_needed_on_this_interpreter(self):
        # Guards the test above against passing for the wrong reason: on an
        # interpreter whose ipaddress lists these ranges as private, the
        # rule in _is_private_ip is what keeps them public. Which ranges
        # are listed depends on the CPython build (6to4 and NAT64
        # local-use are missing from older builds' private table), so
        # the guard needs at least one of them to be listed here and skips
        # otherwise: there the rule is redundant rather than wrong.
        listed = [
            peer
            for peer in (
                "2002:808:808::1",
                "2001:0:4136:e378::1",
                "64:ff9b:1::808:808",
            )
            if ipaddress.ip_address(peer).is_private
        ]
        if not listed:
            pytest.skip("this interpreter lists no transition range as private")
        for peer in listed:
            assert not _is_private_ip(peer)

    def test_neighbours_of_the_transition_ranges_are_unaffected(self):
        # 2001:db8::/32 (documentation) is outside Teredo's 2001::/32 and
        # keeps CPython's private verdict.
        assert _is_private_ip("2001:db8::1")
        assert not _is_private_ip("2001:4860:4860::8888")


class TestConsumerBoundary:
    @pytest.mark.parametrize(
        ("peer", "expected"),
        [
            ("127.0.0.1", True),
            ("8.8.8.8", False),
            # The secure mapped direction, on every interpreter.
            ("::ffff:8.8.8.8", False),
        ],
    )
    def test_classification_is_pure_for_warning_consumers(self, peer, expected):
        # The function is a pure classifier over a string: same input,
        # same answer, no process state (the one-shot warning dedup lives
        # on the middleware instance, not here). This repeated-call pin
        # fails on any change that alters the return value between the
        # two calls with the same input, whatever the source of that
        # change turns out to be.
        assert _is_private_ip(peer) == expected
        assert _is_private_ip(peer) == expected