π Geometric Computer Vision Library for Spatial AI
Python artificial-intelligencecomputer-visiondeep-learninghacktoberfestimage-processingmachine-learningneural-networkpythonpytorchroboticsspatial-ai
fix(geometry): keep cart2pol's radius accurate in half precision and inside eps (#6431)
Debanjan Sarkar814393bf9ff7
diff --git a/changelog.d/6431.fixed.md b/changelog.d/6431.fixed.mdnew file mode 100644index 000000000..d339ffbe6--- /dev/null+++ b/changelog.d/6431.fixed.md@@ -0,0 +1 @@+`cart2pol` now computes float16 and bfloat16 radii in float32 and rounds them once. It used to form `x ** 2 + y ** 2` in the input dtype, so a float16 radius of about 256 or more gave `rho = nan` (`(300, 150)`) and small float16 radii were inexact (`(2e-4, 0)` gave `2.44e-4`). Inside the `eps` neighbourhood `rho` is now the exact radius: it used to be computed as `(rho + 1) - 1`, which in float32 turned `(1e-5, 0)` into `1.0014e-5` and every radius below about `6e-8` into `0`, and in bfloat16 every radius up to `1e-4` into `0`. A float32 or float64 square sum that overflows now gives `rho = inf` instead of `nan`. Elsewhere float32 and float64 values and gradients are unchanged; float16 and bfloat16 radii and gradients can move by about one unit in the last place, to the correctly rounded value. The gradient inside `eps` is still zero, and for float16 input `eps` is compared with the float32 square sum, so radii between `1e-4` and about `2e-4` now get the gradient `x / rho` instead of `0` or a wrong value.diff --git a/kornia/geometry/conversions.py b/kornia/geometry/conversions.pyindex 10befcd4e..1dc44c0d5 100644--- a/kornia/geometry/conversions.py+++ b/kornia/geometry/conversions.py@@ -195,11 +195,9 @@ def pol2cart(rho: torch.Tensor, phi: torch.Tensor) -> tuple[torch.Tensor, torch.``x, y = pol2cart(rho, phi)`` takes radii and angles in radians and returns two tensors, ``x = rho * cos(phi)`` and``y = rho * sin(phi)``, in the broadcast shape of ``rho`` and ``phi``. It undoes :func:`cart2pol`:- ``pol2cart(*cart2pol(x, y))`` returns ``(x, y)`` to rounding as long as ``x ** 2 + y ** 2`` is above the ``eps``- of :func:`cart2pol` and neither overflows nor underflows in the working dtype. Inside the ``eps`` neighbourhood of- the origin :func:`cart2pol` rounds the radius, so in ``float32`` ``(1e-5, 2e-5)`` comes back as- ``(1.0023e-5, 2.0045e-5)`` and ``(3e-8, 4e-8)`` as ``(0, 0)``- (`#6440 <https://github.com/kornia/kornia/issues/6440>`_).+ ``pol2cart(*cart2pol(x, y))`` returns ``(x, y)`` to rounding, near the origin included (in ``float32``+ ``(3e-8, 4e-8)`` comes back as ``(3e-8, 4e-8)``), as long as :func:`cart2pol` can form ``x ** 2 + y ** 2``: it+ does so in ``float32`` for ``float16`` and ``bfloat16`` input and in the input dtype otherwise.Convention:- ``rho = 5``, ``phi = atan2(4, 3)`` gives ``(3., 4.)``, and ``rho = 2``, ``phi = pi / 6`` gives@@ -235,30 +233,26 @@ def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[torr"""Convert cartesian coordinates (x, y) to polar coordinates (rho, phi), phi in radians.``rho, phi = cart2pol(x, y)`` returns two tensors in the broadcast shape of ``x`` and ``y``: the radius- ``sqrt(x ** 2 + y ** 2)`` and the angle ``atan2(y, x)`` in radians, in :math:`[-\pi, \pi]`. The radius is- computed from the unscaled ``x ** 2 + y ** 2``, so it is ``nan`` once that overflows- (`#6414 <https://github.com/kornia/kornia/issues/6414>`_). ``eps`` keeps the gradient of ``rho`` finite at the- origin, but it also rounds every radius with ``x ** 2 + y ** 2 <= eps``, as the convention below details- (`#6440 <https://github.com/kornia/kornia/issues/6440>`_). For the reverse, use :func:`pol2cart`.+ ``sqrt(x ** 2 + y ** 2)`` and the angle ``atan2(y, x)`` in radians, in :math:`[-\pi, \pi]`. ``float16`` and+ ``bfloat16`` radii are computed in ``float32`` and rounded once; ``float32`` and ``float64`` radii are computed+ from the unscaled square sum in the input dtype, so they are ``inf`` once it overflows. ``eps`` keeps the gradient+ of ``rho`` finite at the origin without changing its value. For the reverse, use :func:`pol2cart`.Convention:- ``(3., 4.)`` gives ``(5., 0.9273)``- - ``x ** 2 + y ** 2`` overflows from a radius of about ``256`` in- ``float16`` (``(300., 150.)`` gives ``rho = nan``), ``1.8e19`` in+ - every ``float16`` radius is correctly rounded, since ``float32``+ holds the square of every ``float16`` value: ``(300., 150.)`` gives+ ``335.5``+ - the square sum overflows from a radius of about ``1.8e19`` in``float32`` and ``bfloat16`` and ``1.3e154`` in ``float64``, and- ``rho`` is then ``nan``, not ``inf``- (`#6414 <https://github.com/kornia/kornia/issues/6414>`_)- - where ``x ** 2 + y ** 2 <= eps`` (radii up to about ``1e-4`` at the- default ``eps``), ``rho`` is computed as ``(rho + 1) - 1``, which- rounds it to a multiple of the spacing of the dtype at ``1``- (`#6440 <https://github.com/kornia/kornia/issues/6440>`_): in- ``float32`` ``(1e-5, 0.)`` gives ``1.0014e-5`` and every radius below- about ``6e-8``, ``(1e-20, 0.)`` included, gives ``0``; in ``bfloat16``- every radius up to ``1e-4`` gives ``0``; in ``float64`` ``(1e-14, 0.)``- gives ``9.992e-15``. In ``float16`` no nonzero sum is that small, so- only a sum of ``0`` takes that branch; there a sum that underflows- makes ``rho`` inexact or ``0`` instead: ``(2e-4, 0.)`` gives- ``2.44e-4``+ ``rho`` is then ``inf``, as for an infinite coordinate. Below a+ radius of about ``1e-19`` and ``1e-154`` it underflows and ``rho``+ loses precision: in ``float32`` ``(1e-20, 0.)`` gives+ ``9.99997e-21`` and ``(1e-25, 0.)`` gives ``0``+ - inside the ``eps`` neighbourhood, ``x ** 2 + y ** 2 <= eps`` (radii+ up to ``1e-4`` at the default ``eps``), the gradient of ``rho`` is+ ``0`` and its value is still the radius: in ``float32``+ ``(1e-5, 0.)`` gives ``1e-5``- ``phi`` is ``atan2(y, x)`` in **radians**, so it lies in:math:`[-\pi, \pi]`, is ``0`` on the ``+x`` axis and grows toward``+y`` (``x = 0``, ``y = 1`` gives ``phi = 1.5708``). Both endpoints@@ -279,9 +273,9 @@ def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[tory: the second coordinates.eps: squared-radius threshold: at or below it the gradient of ``rho``is zero, which avoids the ``0 / 0`` of the exact gradient at the- origin, and ``rho`` is rounded as described above- (`#6440 <https://github.com/kornia/kornia/issues/6440>`_). The- default ``1e-8`` underflows in ``float16``.+ origin. It does not change the value of ``rho``. It is compared+ with the square sum in the dtype that sum is formed in, ``float32``+ for ``float16`` and ``bfloat16`` input.Returns:- rho: the radius, in the broadcast shape of ``x`` and ``y``.@@ -300,11 +294,18 @@ def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[torif not (isinstance(x, torch.Tensor) & isinstance(y, torch.Tensor)):raise TypeError(f"Input type is not a torch.Tensor. Got {type(x)}, {type(y)}")- squared_radius = x**2 + y**2- safe_squared_radius = torch.where(squared_radius > eps, squared_radius, torch.ones_like(squared_radius))- safe_rho = torch.sqrt(safe_squared_radius)- rho = torch.sqrt(squared_radius).detach() + safe_rho - safe_rho.detach()phi = torch.atan2(y, x)+ # phi has the dtype of rho under torch's rank-aware promotion, which promote_types(x.dtype, y.dtype) ignores+ half = phi.dtype in (torch.float16, torch.bfloat16)+ # float32 holds the square of every float16 value, so half-precision radii neither overflow nor lose small values+ xw, yw = (x.float(), y.float()) if half else (x, y)+ squared_radius = xw**2 + yw**2+ outside = squared_radius > eps+ # inside eps, the exact radius from a detached root keeps the zero gradient at the origin+ safe_rho = torch.sqrt(torch.where(outside, squared_radius, torch.ones_like(squared_radius)))+ rho = torch.where(outside, safe_rho, torch.sqrt(squared_radius).detach())+ if half:+ rho = rho.to(phi.dtype)return rho, phidiff --git a/tests/geometry/test_conversions.py b/tests/geometry/test_conversions.pyindex 61db4c1a0..b286aa3f0 100644--- a/tests/geometry/test_conversions.py+++ b/tests/geometry/test_conversions.py@@ -2605,20 +2605,11 @@ class TestPolCartConversions(BaseTester):rho_origin = kornia.geometry.conversions.cart2pol(zero, zero)[0]self.assert_close(rho_origin, zero, atol=0.0, rtol=0.0)- # Use float64 for the tiny-radius regression where supported so the- # sqrt(x**2) round-trip is not dominated by float32 precision.- tiny_dtype = torch.float32 if device.type == "mps" else torch.float64- tiny_rtol = 2e-3 if tiny_dtype == torch.float32 else 1e-12- tiny_atol = 1e-7 if tiny_dtype == torch.float32 else 1e-15-- x = torch.tensor(5.0e-5, device=device, dtype=tiny_dtype)- y = torch.tensor(0.0, device=device, dtype=tiny_dtype)- rho_sub_eps = kornia.geometry.conversions.cart2pol(x, y)[0]- self.assert_close(rho_sub_eps, x, rtol=tiny_rtol, atol=tiny_atol)-- x = torch.tensor(1.0e-4, device=device, dtype=tiny_dtype)- rho_at_eps = kornia.geometry.conversions.cart2pol(x, y)[0]- self.assert_close(rho_at_eps, x, rtol=tiny_rtol, atol=tiny_atol)+ # inside the eps neighbourhood and at its edge the radius is |x| exactly, in the test dtype; it used to be+ # (rho + 1) - 1, which this test tolerated with float64 and rtol=2e-3 (#6440)+ x = torch.tensor([5.0e-5, 1.0e-4], device=device, dtype=dtype)+ rho_sub_eps = kornia.geometry.conversions.cart2pol(x, torch.zeros_like(x))[0]+ self.assert_close(rho_sub_eps, x, atol=0.0, rtol=0.0)x = torch.tensor(3.0, device=device, dtype=dtype)y = torch.tensor(4.0, device=device, dtype=dtype)@@ -2633,6 +2624,136 @@ class TestPolCartConversions(BaseTester):self.assert_close(rho, torch.tensor(0.0, device=device, dtype=torch.float32))self.assert_close(xy.grad, torch.zeros_like(xy))+ @pytest.mark.parametrize(+ "rho_dtype, x, y",+ [+ # a float16 x**2 + y**2 overflows from a radius of about 256 (#6414); float32 holds it+ (torch.float16, 300.0, 150.0),+ (torch.float16, 60000.0, 30000.0),+ # a float16 x**2 + y**2 underflows: the radius came back inexact+ (torch.float16, 2e-4, 0.0),+ ],+ )+ def test_cart2pol_rho_is_correctly_rounded_when_square_sum_is_not_representable(self, rho_dtype, x, y, device):+ # Expected: the float64 hypot of the dtype-rounded inputs, rounded once to the dtype.+ xt = torch.tensor(x, device=device, dtype=rho_dtype)+ yt = torch.tensor(y, device=device, dtype=rho_dtype)+ # rounded to rho_dtype on the CPU before it moves, so no float64 tensor reaches the device (MPS has none)+ expected = torch.tensor(math.hypot(xt.item(), yt.item()), dtype=torch.float64).to(rho_dtype).to(device)++ rho = kornia.geometry.conversions.cart2pol(xt, yt)[0]++ self.assert_close(rho, expected, atol=0.0, rtol=0.0)++ def test_cart2pol_float16_rho_with_a_0d_float32_operand_is_formed_in_float32(self, device):+ # rho is float16 under torch's rank-aware promotion, so its square sum must still be formed in float32 (#6414):+ # promote_types(float16, float32) would say float32 and skip that+ x = torch.tensor([300.0], device=device, dtype=torch.float16)+ y = torch.tensor(150.0, device=device, dtype=torch.float32)++ rho = kornia.geometry.conversions.cart2pol(x, y)[0]++ assert rho.dtype == torch.float16+ self.assert_close(rho, torch.tensor([335.5], device=device, dtype=torch.float16), atol=0.0, rtol=0.0)++ @pytest.mark.parametrize("radius", [1e-4, 5e-5, 1e-5, 1e-6, 5e-8])+ def test_cart2pol_rho_is_exact_inside_eps_6440(self, radius, device, dtype):+ # Inside the eps neighbourhood rho was computed as (rho + 1) - 1, which rounded it to a multiple of the+ # spacing at 1 (#6440); it is now the exact square root, sqrt(x ** 2) = |x|.+ x = torch.tensor([radius, -radius], device=device, dtype=dtype)++ rho = kornia.geometry.conversions.cart2pol(x, torch.zeros_like(x))[0]++ self.assert_close(rho, x.abs(), atol=0.0, rtol=0.0)++ def test_cart2pol_rho_gradient_is_finite_for_a_large_radius(self, device):+ # d rho / d (x, y) = (x, y) / rho = (2, 1) / sqrt(5), also where the float16 x**2 + y**2 overflows.+ rho_dtype = torch.float16+ xy = torch.tensor([300.0, 150.0], device=device, dtype=rho_dtype, requires_grad=True)++ kornia.geometry.conversions.cart2pol(xy[0], xy[1])[0].backward()++ expected = torch.tensor([2.0, 1.0], device=device, dtype=torch.float32) / math.sqrt(5.0)+ self.assert_close(xy.grad.float(), expected, atol=1e-2, rtol=1e-2)++ @pytest.mark.parametrize("rho_dtype", [torch.float16, torch.bfloat16, torch.float32])+ def test_cart2pol_rho_gradient_does_not_overflow_for_a_large_upstream_gradient(self, rho_dtype, device):+ # The backward pass must not form upstream * x before dividing by rho: in float16 a 30000 upstream+ # gradient times x = 3 overflows although the result, 30000 * (0.6, 0.8), is representable.+ xy = torch.tensor([3.0, 4.0], device=device, dtype=rho_dtype, requires_grad=True)++ rho = kornia.geometry.conversions.cart2pol(xy[0], xy[1])[0]+ rho.backward(torch.tensor(30000.0, device=device, dtype=rho_dtype))++ expected = torch.tensor([18000.0, 24000.0], device=device, dtype=torch.float32)+ self.assert_close(xy.grad.float(), expected, atol=0.0, rtol=1e-2)++ @pytest.mark.parametrize(+ "x_dtype, x_shape, y_dtype, y_shape, expected",+ [+ (torch.float32, (3,), torch.float64, (), torch.float32),+ (torch.float16, (3,), torch.float32, (), torch.float16),+ (torch.float64, (), torch.float32, (3,), torch.float32),+ (torch.int64, (3,), torch.int64, (3,), None), # None: the default float dtype+ ],+ )+ def test_cart2pol_rho_dtype_follows_torch_promotion_like_phi(+ self, x_dtype, x_shape, y_dtype, y_shape, expected, device+ ):+ # rho has the dtype of sqrt(x**2 + y**2) under torch's rank-aware promotion, as phi = atan2(y, x) does: a 0-d+ # operand does not decide it, and integer input gives the default float dtype.+ if device.type == "mps" and torch.float64 in (x_dtype, y_dtype):+ pytest.skip("MPS does not support float64")+ expected = torch.get_default_dtype() if expected is None else expected+ x = torch.full(x_shape, 3, device=device, dtype=x_dtype)+ y = torch.full(y_shape, 4, device=device, dtype=y_dtype)++ rho, phi = kornia.geometry.conversions.cart2pol(x, y)++ assert rho.dtype == phi.dtype == expected+ self.assert_close(rho, torch.full_like(rho, 5.0))++ def test_cart2pol_torchscript_matches_eager(self, device, dtype):+ # cart2pol is reached from scripted code (kornia.feature.mkd), so it must stay scriptable. (300, 150) and+ # (1e4, 5e3) overflow a float16 square sum, 5e-5 lies inside eps, and (0, 0) is the origin.+ x = torch.tensor([3.0, 0.0, 300.0, 1e4, 5e-5], device=device, dtype=dtype)+ y = torch.tensor([4.0, 0.0, 150.0, 5e3, 0.0], device=device, dtype=dtype)+ scripted = torch.jit.script(kornia.geometry.conversions.cart2pol)++ rho, phi = scripted(x, y)++ expected_rho, expected_phi = kornia.geometry.conversions.cart2pol(x, y)+ self.assert_close(rho, expected_rho, atol=0.0, rtol=0.0)+ self.assert_close(phi, expected_phi, atol=0.0, rtol=0.0)++ @pytest.mark.device_agnostic+ def test_cart2pol_onnx_export_legacy_matches_eager(self, dtype):+ # cart2pol must stay exportable: 5e-5 takes the inside-eps branch and (0, 0) is the origin, the rest the+ # outside one.+ if dtype != torch.float32:+ pytest.skip("the exported graph is checked once, in float32")+ pytest.importorskip("onnx")+ ort = pytest.importorskip("onnxruntime")++ class Cart2Pol(torch.nn.Module):+ def forward(self, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:+ return torch.stack(kornia.geometry.conversions.cart2pol(x, y), -1)++ x = torch.tensor([3.0, 0.0, 300.0, 5e-5, -2.0], dtype=dtype)+ y = torch.tensor([4.0, 0.0, 150.0, 0.0, -1.0], dtype=dtype)+ model = Cart2Pol()+ buffer = io.BytesIO()+ with warnings.catch_warnings():+ warnings.simplefilter("ignore")+ torch.onnx.export(model, (x, y), buffer, input_names=["x", "y"], opset_version=17, dynamo=False)+ session = ort.InferenceSession(buffer.getvalue(), providers=["CPUExecutionProvider"])+ exported = torch.from_numpy(session.run(None, {"x": x.numpy(), "y": y.numpy()})[0])+ eager = model(x, y)+ self.assert_close(exported[:, 0], eager[:, 0])+ # phi is compared off the origin only: the exported atan2(0, 0) is nan, independently of the radius+ off_origin = (x != 0) | (y != 0)+ self.assert_close(exported[off_origin, 1], eager[off_origin, 1])+def test_convention_positive_rotation_decreases_cart2pol_phi(self, device, dtype):# Cross-symbol convention pin: enforces the opposite-sense relation between# angle_to_rotation_matrix and cart2pol stated canonically in cart2pol's Convention