š Geometric Computer Vision Library for Spatial AI
Python artificial-intelligencecomputer-visiondeep-learninghacktoberfestimage-processingmachine-learningneural-networkpythonpytorchroboticsspatial-ai
docs(geometry): rewrite transform, conversions, epipolar, homography, solvers, RANSAC and pose in the choose-then-use style (#6432)
Dmytro Mishkincaa0f394e4c1
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 51ae883fd..fde1d088f 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -203,7 +203,7 @@ The rest of this guide contains practical and technical guidance so nobody has tA docstring is how a reader, human or agent, decides which function to call, so it states the choice before the details. `kornia/geometry/depth.py` is the worked example. Docstrings are reST: write literals with double backticks, as in ``` ``(B, 1, H, W)`` ```, and keep single backticks for roles such as `` :func:`name` ``.- **Title line**: one physical line, in task words, imperative, naming what sets the function apart from its siblings (2D or 3D, channels first or last, the criterion, one call or module form). Aim for about 80 characters; 120 is the hard limit. Avoid markup that breaks reST, such as a bare `*`. No two unrelated public symbols may share one (`tests/test_docstring_first_lines.py`); a class and the function it wraps may.-- **Status words** come first when they apply: `Deprecated: use X.` or `Alias of X.`. Do not label a function "Legacy"; for a nearby better choice, write a neutral "for X, use Y" in the contract paragraph.+- **Status words** come first when they apply: `Deprecated: use X.` or `Alias of X.`. An alias whose whole job is to forward to another object has the title line ``Alias of :func:`target`.``, and a deprecated object the title line ``Deprecated: use :func:`target`.``, with ``:class:`` or another Python-domain role for a target that is not a function; either title needs no `See Also:`. The target is another public kornia object, `Deprecated: use` belongs to objects that `@deprecated` wraps, and `Alias of` to an undecorated function whose body only returns the target called with its own arguments, unchanged, or to a class that subclasses the target and adds nothing. Do not label a function "Legacy"; for a nearby better choice, write a neutral "for X, use Y" in the contract paragraph.- **Contract paragraph** (2-5 sentences) starts from the call: ``` ``points = depth_to_3d(depth, camera_matrix)`` ``` takes ... and returns .... A callable module shows construction and invocation; a data class (`Boxes`, `PinholeCamera`) shows construction and its main method. Name inputs and output with shapes, frames and units, and end with "use X instead" when a nearby task is better served elsewhere.- **Existing blocks** stay: `.. note::`, `Convention:`, `Args:`, `Return:`/`Returns:`, `Raises:`, trimmed where the paragraph already says the same. In a long class docstring, put whatever decides correct use in the first paragraph.- **`See Also:`** goes after `Returns`/`Raises` and before `Example`.@@ -216,7 +216,8 @@ A docstring is how a reader, human or agent, decides which function to call, soWhat `tests/test_docstring_style.py` checks:- every package of `kornia` is assigned to in scope or excluded (global);-- each public function and class of a converted module has a `See Also:` whose entries carry an explanation and name a kornia object, with no self-link or private target; an exception needs a non-empty reason;+- a converted module exports no function or class without a docstring;+- each public function and class of a converted module has a `See Also:` whose entries carry an explanation and name a kornia object, with no self-link or private target; an exception needs a non-empty reason, and an ``Alias of :func:`target`.`` or ``Deprecated: use :func:`target`.`` title line replaces the `See Also:`;- every Python-domain object role (`:func:`, `:class:`, `:meth:`, `:attr:`, `:data:`, `:const:`, `:exc:`, `:obj:`) in a converted object's docstring resolves as Sphinx resolves it. It must name either an object registered by the `.rst` pages under `docs/source`, or a torch, Python or NumPy object in the cached intersphinx inventories (`docs/source/_intersphinx`). Registered objects include the data attributes autodoc documents: enum members and dataclass fields under `:undoc-members:`, and variables with a `#:` comment or a docstring. An intersphinx object must be of a type the role links to: a function for `:func:`, a method for `:meth:`, an attribute or property for `:attr:`, and nothing for `:const:`. Only `:func:` and `:meth:` drop a trailing `()`. Roles in a catalog module docstring must import;- no `kornia/**/*.py` file or `docs/source` page writes a shape tuple as `:math:`, comments included (global); a tuple that reads as a shape but is a value is listed with what it is in `MATH_NON_SHAPES` (`testing/docstring_rollout.py`), and an entry that no longer occurs fails;- a catalog namespace's module docstring names every export of its `__all__` inside a role or a double-backtick literal, where a qualified name must be a path to that export (`torch.nn.functional.normalize` does not catalog `kornia.enhance.normalize`); the same word in plain prose or in single backticks does not count.diff --git a/docs/source/geometry.conversions.rst b/docs/source/geometry.conversions.rstindex 9b7b16cff..4cd7e110d 100644--- a/docs/source/geometry.conversions.rst+++ b/docs/source/geometry.conversions.rst@@ -6,9 +6,22 @@ kornia.geometry.conversions.. currentmodule:: kornia.geometry.conversions+The functions on this page convert between geometric representations: Euclidean and homogeneous points, pixel+and normalized :math:`[-1, 1]` coordinates, pixel and normalized homographies, the forms of a 3D rotation, and pose+matrices in different camera conventions. Quaternions are ``(w, x, y, z)``, real part first, in the Hamilton+convention; axis-angle vectors and Euler angles are in radians, and :func:`angle_to_rotation_matrix` takes degrees.+The :math:`[-1, 1]` normalizations are corner-aligned unless an ``align_corners=False`` argument selects the+half-pixel convention, which only the 2D matrices and homographies offer+(`#4503 <https://github.com/kornia/kornia/issues/4503>`_). See :doc:`Conventions & Pitfalls </get-started/conventions>`+and :doc:`camera and world conventions </get-started/camera-conventions>`.+Angles------+Convert angles between radians and degrees (:func:`rad2deg`, :func:`deg2rad`) and points between polar and cartesian+coordinates (:func:`pol2cart`, :func:`cart2pol`), and build a 2D rotation matrix from an angle in degrees+(:func:`angle_to_rotation_matrix`).+.. autofunction:: rad2deg.. autofunction:: deg2rad.. autofunction:: pol2cart@@ -18,6 +31,14 @@ AnglesCoordinates-----------+Append or divide out the homogeneous coordinate of ``(*, N, D)`` points (:func:`convert_points_to_homogeneous`,+:func:`convert_points_from_homogeneous`), extend an affine matrix to a homography+(:func:`convert_affinematrix_to_homography`, :func:`convert_affinematrix_to_homography3d`), map pixel coordinates to+:math:`[-1, 1]` and back (:func:`normalize_pixel_coordinates`, :func:`denormalize_pixel_coordinates`, and in 3D+:func:`normalize_pixel_coordinates3d`, :func:`denormalize_pixel_coordinates3d`, which read ``(d, x, y)``), and map+pixels to normalized camera coordinates with the intrinsics and back (:func:`normalize_points_with_intrinsics`,+:func:`denormalize_points_with_intrinsics`).+.. autofunction:: convert_points_from_homogeneous.. autofunction:: convert_points_to_homogeneous.. autofunction:: convert_affinematrix_to_homography@@ -33,6 +54,11 @@ CoordinatesHomography----------+Express a sourceādestination pixel homography in normalized :math:`[-1, 1]` coordinates and back+(:func:`normalize_homography`, :func:`denormalize_homography`, and in 3D :func:`normalize_homography3d`,+:func:`denormalize_homography3d`), or get the normalization matrix itself (:func:`normal_transform_pixel`,+:func:`normal_transform_pixel3d`).+.. autofunction:: normalize_homography.. autofunction:: denormalize_homography.. autofunction:: normalize_homography3d@@ -43,6 +69,12 @@ HomographyQuaternion----------+Convert ``(w, x, y, z)`` quaternions to axis-angle vectors and rotation matrices (:func:`quaternion_to_axis_angle`,+:func:`quaternion_to_rotation_matrix`), apply the exponential and logarithm maps, whose 3-vector is half the+axis-angle vector (:func:`quaternion_log_to_exp`, :func:`quaternion_exp_to_log`), and normalize to unit length+(:func:`normalize_quaternion`). For a quaternion object that composes and inverts, see+:class:`~kornia.geometry.quaternion.Quaternion`.+.. autofunction:: quaternion_to_axis_angle.. autofunction:: quaternion_to_rotation_matrix.. autofunction:: quaternion_log_to_exp@@ -52,29 +84,48 @@ QuaternionVector------+Build the skew-symmetric cross-product matrix of a 3-vector (:func:`vector_to_skew_symmetric_matrix`).+.. autofunction:: vector_to_skew_symmetric_matrixRotation Matrix---------------+Convert rotation matrices to axis-angle vectors (:func:`rotation_matrix_to_axis_angle`) and to quaternions+(:func:`rotation_matrix_to_quaternion`).+.. autofunction:: rotation_matrix_to_axis_angle.. autofunction:: rotation_matrix_to_quaternionAxis Angle----------+Convert axis-angle vectors, the unit axis scaled by the angle in radians, to quaternions+(:func:`axis_angle_to_quaternion`) and to rotation matrices (:func:`axis_angle_to_rotation_matrix`).+.. autofunction:: axis_angle_to_quaternion.. autofunction:: axis_angle_to_rotation_matrixEuler Angles------------+Convert roll, pitch and yaw in radians, for the rotation ``Rz(yaw) @ Ry(pitch) @ Rx(roll)``, to a quaternion and+back, with the quaternion given as four separate tensors (:func:`quaternion_from_euler`,+:func:`euler_from_quaternion`).+.. autofunction:: quaternion_from_euler.. autofunction:: euler_from_quaternionPose----------+Pack and split ``(B, 4, 4)`` pose matrices (:func:`Rt_to_matrix4x4`, :func:`matrix4x4_to_Rt`), invert a rigid+``(R, t)`` pose between camera-to-world and world-to-camera (:func:`camtoworld_to_worldtocam_Rt`,+:func:`worldtocam_to_camtoworld_Rt`), switch a camera-to-world pose between the OpenGL and OpenCV camera axes+(:func:`camtoworld_graphics_to_vision_4x4`, :func:`camtoworld_graphics_to_vision_Rt`,+:func:`camtoworld_vision_to_graphics_4x4`, :func:`camtoworld_vision_to_graphics_Rt`), and convert quaternion poses+between ARKit and Colmap (:func:`ARKitQTVecs_to_ColmapQTVecs`, :func:`ColmapQTVecs_to_ARKitQTVecs`).+.. autofunction:: Rt_to_matrix4x4.. autofunction:: matrix4x4_to_Rt.. autofunction:: worldtocam_to_camtoworld_Rtdiff --git a/docs/source/geometry.epipolar.rst b/docs/source/geometry.epipolar.rstindex e688b47c7..0cb26334f 100644--- a/docs/source/geometry.epipolar.rst+++ b/docs/source/geometry.epipolar.rst@@ -6,7 +6,14 @@ kornia.geometry.epipolar.. currentmodule:: kornia.geometry.epipolar-Functions for epipolar geometry, as used in Structure from Motion (SfM) and two-view pose estimation.+Functions for two-view epipolar geometry, as used in Structure from Motion (SfM) and two-view pose estimation:+estimate fundamental and essential matrices, recover the relative pose, score correspondences and triangulate. The+first image's points come first, and both matrices satisfy :math:`x_2^\top F x_1 = 0`, ``F`` in pixels and ``E`` in+normalized camera coordinates :math:`K^{-1} [u, v, 1]^\top`. Camera poses are world-to-camera extrinsics, and the+relative motion ``(R, t)`` of camera 2 to camera 1 takes camera-1 coordinates to camera-2 coordinates,+:math:`X_2 = R X_1 + t`. See :ref:`two-view geometry <two-view-conventions>` for a comparison with OpenCV. For a+plane or a pure rotation use :doc:`homographies <geometry.homography>`, and with outliers+:class:`~kornia.geometry.ransac.RANSAC`... image:: data/epipolar_geometry.svg.png:alt: Epipolar geometry between two views@@ -14,6 +21,14 @@ Functions for epipolar geometry, as used in Structure from Motion (SfM) and two-Essential---------++Estimate up to ten essential matrices from calibrated correspondences (:func:`find_essential`), convert from and to+other models (:func:`essential_from_fundamental`, :func:`essential_from_Rt`, :func:`project_to_essential`), and+recover the relative pose: the four candidates (:func:`motion_from_essential`) or the two rotations and the+translation that combine to them (:func:`decompose_essential_matrix`, :func:`decompose_essential_matrix_no_svd`), the+one with the points in front of both cameras (:func:`motion_from_essential_choose_solution`), or the motion between+two known poses (:func:`relative_camera_motion`).+.. autofunction:: find_essential.. autofunction:: project_to_essential.. autofunction:: essential_from_fundamental@@ -27,6 +42,13 @@ EssentialFundamental-----------+Estimate the fundamental matrix from pixel correspondences with the 8-point or 7-point algorithm+(:func:`find_fundamental`), convert from an essential matrix or two projection matrices+(:func:`fundamental_from_essential`, :func:`fundamental_from_projections`), compute epipolar lines and the nearest+points on them (:func:`compute_correspond_epilines`, :func:`get_closest_point_on_epipolar_line`,+:func:`get_perpendicular`), and normalize points and matrices for the estimators (:func:`normalize_points`,+:func:`normalize_transformation`).+.. autofunction:: find_fundamental.. autofunction:: fundamental_from_essential.. autofunction:: fundamental_from_projections@@ -40,6 +62,11 @@ FundamentalMetrics-------+Score correspondences against a fundamental matrix: the first-order geometric error (:func:`sampson_epipolar_distance`),+the summed squared distances of both points to their epipolar lines (:func:`symmetrical_epipolar_distance`), or the+distance in one image only (:func:`left_to_right_epipolar_distance` in image 2,+:func:`right_to_left_epipolar_distance` in image 1).+.. autofunction:: sampson_epipolar_distance.. autofunction:: symmetrical_epipolar_distance.. autofunction:: left_to_right_epipolar_distance@@ -48,6 +75,11 @@ MetricsProjection----------+Compose and decompose projection matrices ``P = K [R | t]`` (:func:`projection_from_KRt`,+:func:`KRt_from_projection`), get a canonical pair for a fundamental matrix (:func:`projections_from_fundamental`)+and the depth of points in a camera (:func:`depth_from_point`), and build camera matrices+(:func:`intrinsics_like`, :func:`scale_intrinsics`, :func:`random_intrinsics`).+.. autofunction:: projection_from_KRt.. autofunction:: KRt_from_projection.. autofunction:: depth_from_point@@ -59,11 +91,16 @@ ProjectionNumeric-------+The skew-symmetric matrix :math:`[x]_\times` of 3-vectors with any number of leading dimensions.+.. autofunction:: cross_product_matrixTriangulation-------------+Triangulate 3D points from two projection matrices and matched points (:func:`triangulate_points`), and generate a+random synthetic scene to test the estimators on (:func:`generate_scene`).+.. autofunction:: triangulate_points.. autofunction:: generate_scenediff --git a/docs/source/geometry.homography.rst b/docs/source/geometry.homography.rstindex 8c6987224..e0470c74e 100644--- a/docs/source/geometry.homography.rst+++ b/docs/source/geometry.homography.rst@@ -4,7 +4,5 @@ kornia.geometry.homography.. meta:::description: The kornia.geometry.homography module provides essential tools for manipulating and working with homographies, which describe the transformation between two images of the same scene from different viewpoints. The module includes a variety of functions for computing, applying, and manipulating homographies, making it useful for tasks such as image stitching, object tracking, and perspective warping.-Functions to estimate homographies from point or line correspondences and to score them.-.. automodule:: kornia.geometry.homography:members:diff --git a/docs/source/geometry.pose.rst b/docs/source/geometry.pose.rstindex bfd441a5f..30cf5ec90 100644--- a/docs/source/geometry.pose.rst+++ b/docs/source/geometry.pose.rst@@ -6,6 +6,11 @@ kornia.geometry.pose.. currentmodule:: kornia.geometry.pose+:class:`NamedPose` holds an :class:`~kornia.geometry.liegroup.Se2` or :class:`~kornia.geometry.liegroup.Se3` pose+together with the names of its source and destination frames. A pose ``b_from_a`` maps points in frame ``a`` to+frame ``b``, and ``c_from_b * b_from_a`` is ``c_from_a``: a composition whose frames do not chain raises instead of+returning a wrong pose.+.. autoclass:: NamedPose:members::special-members: __init__, __mul__diff --git a/docs/source/geometry.ransac.rst b/docs/source/geometry.ransac.rstindex 8b698b10a..00c53e710 100644--- a/docs/source/geometry.ransac.rst+++ b/docs/source/geometry.ransac.rst@@ -6,7 +6,15 @@ kornia.geometry.ransac.. currentmodule:: kornia.geometry.ransac-Robust model fitting with RANSAC (Random Sample Consensus), used to estimate affine transforms, homographies, fundamental and essential matrices from noisy correspondences.+Robust model fitting with RANSAC (Random Sample Consensus), used to estimate affine transforms, homographies,+fundamental and essential matrices from noisy correspondences. ``RANSAC(model_type, inl_th)`` builds the estimator,+and calling it on one set of ``(N, 2)`` correspondences (``(N, 2, 2)`` segments for+``"homography_from_linesegments"``) of the first and the second image returns the ``(3, 3)`` model and the ``(N,)``+inlier mask. A homography or affine map takes the first image to the second, a fundamental matrix satisfies+:math:`x_2^\top F x_1 = 0`, and an essential matrix expects camera-normalized coordinates; ``inl_th`` is a distance in+the units of the correspondences. For correspondences known to be inliers, call the estimators+directly: :func:`~kornia.geometry.homography.find_homography_dlt`, :func:`~kornia.geometry.solvers.find_affine`,+:func:`~kornia.geometry.epipolar.find_fundamental` and :func:`~kornia.geometry.epipolar.find_essential`.Affine transforms-----------------diff --git a/docs/source/geometry.rst b/docs/source/geometry.rstindex d2568776a..b051c6f6a 100644--- a/docs/source/geometry.rst+++ b/docs/source/geometry.rst@@ -6,25 +6,32 @@ kornia.geometryGeometric image transformations are another key ingredient in computer vision for manipulating images.Since geometry operations are typically performed in 2D or 3D, we provide algorithms for both cases.-This module, the original core of the library, consists of the following main submodules:+This module, the original core of the library, groups its submodules by task:-- :doc:`transform <geometry.transform>`: low-level interfaces to manipulate 2D images, with routines for rotating,- scaling, translating and shearing; cropping functions in several modalities such as central crops and- crop-and-resize; vertical and horizontal flips; resizing; functions to warp tensors given affine or- perspective transformations; and utilities to compute the transformation matrices behind those operations.-- :doc:`camera <geometry.camera>`: routines specific to different camera representations, such as the pinhole- or orthographic models, including projecting and unprojecting points between the camera and the world frame.-- :doc:`conversions <geometry.conversions>`: conversions between angle representations (radians and degrees),- coordinate normalization, and homogeneous to Euclidean coordinates. It also includes conversions between 3D- rotation representations: quaternions, axis-angle, rotation matrices and Euler angles.-- :doc:`linalg <geometry.linalg>`: general rigid-body homogeneous transformations, with functions to- transform points between frames and to compose, invert and compute relative transformations.-- :doc:`depth <geometry.depth>`: layers to manipulate depth maps, such as computing 3D point clouds from depth maps- and calibrated cameras, per-pixel surface normals, and warping frames given a calibrated camera setup.+- Warp, resize, crop, flip or register images, and build the matrices of those warps:+ :doc:`transform <geometry.transform>`.+- Convert between homogeneous and Euclidean points, pixel and normalized coordinates, rotation forms and pose+ conventions: :doc:`conversions <geometry.conversions>`.+- Apply a homogeneous transformation to points, compose, invert or relate ``4 x 4`` transformations, and measure+ distances of points to points and lines: :doc:`linalg <geometry.linalg>`. Grids of pixel coordinates:+ :doc:`grid <geometry.grid>`.+- Camera models, and the projection and unprojection of points: :doc:`camera <geometry.camera>`; lens distortion and+ pose from 2D-3D correspondences: :doc:`calibration <geometry.calibration>`.+- 3D points and surface normals from depth maps, and images warped by depth: :doc:`depth <geometry.depth>`.+- Two-view geometry, fundamental and essential matrices, relative pose and triangulation:+ :doc:`epipolar <geometry.epipolar>`; homographies from points or line segments:+ :doc:`homography <geometry.homography>`; any of these models robust to outliers: :doc:`ransac <geometry.ransac>`;+ the affine, null-vector and polynomial solvers they build on: :doc:`solvers <geometry.solvers>`.+- Rotations and rigid motions as objects that compose and invert: :doc:`liegroup <geometry.liegroup>`,+ :doc:`quaternion <geometry.quaternion>`, and poses between named frames, :doc:`pose <geometry.pose>`.+- Geometric primitives: 2D and 3D boxes, :doc:`boxes <geometry.boxes>`, with the tensor functions of+ :doc:`bbox <geometry.bbox>`; :doc:`keypoints <geometry.keypoints>`; lines and planes,+ :doc:`line <geometry.line>`; 2D and 3D vectors, :doc:`vector <geometry.vector>`.+- Sub-pixel peaks, soft argmax and non-maxima suppression of response maps: :doc:`subpix <geometry.subpix>`.+- Point clouds to and from PLY files: :doc:`pointcloud <geometry.pointcloud>`.-The remaining submodules, listed below, cover bounding boxes and keypoints, calibration, epipolar geometry,-homographies, Lie groups, named poses, lines, point clouds, quaternions, RANSAC, polynomial solvers and sub-pixel-refinement.+``kornia.geometry`` re-exports the names of every submodule here except ``boxes``, ``keypoints`` and ``vector``+(import those from their modules, such as ``kornia.geometry.boxes.Boxes``), and of ``quaternion`` only ``Quaternion``... admonition:: About ``align_corners``diff --git a/docs/source/geometry.solvers.rst b/docs/source/geometry.solvers.rstindex 21353fdf9..762471336 100644--- a/docs/source/geometry.solvers.rst+++ b/docs/source/geometry.solvers.rst@@ -6,7 +6,8 @@ kornia.geometry.solvers.. currentmodule:: kornia.geometry.solvers-Affine, polynomial and homogeneous linear solvers used by the geometric estimators.+Solvers that the geometric estimators build on: a weighted affine fit, the null vector of a rank-3 matrix without an+SVD, and the real roots of quadratic, cubic and quartic polynomials.Affine estimation-----------------@@ -22,6 +23,9 @@ weighted least-squares fitting and returns a source-to-target homogeneousHomogeneous Solvers-------------------+:func:`null_vector_3x4` returns the null vector of a rank-3 ``(*, 3, 4)`` matrix from its ``3 x 3`` cofactors,+unnormalized, without a LAPACK or cuSOLVER call; a matrix of lower rank gives the zero vector.+.. autofunction:: null_vector_3x4Polynomial Solvers@@ -36,7 +40,9 @@ is defined by the represented coefficients. See each function's precision andgradient conventions. Under ``torch.compile`` with Inductor on CUDA, fusedmultiply-adds can lose exact double roots; this known limitation is not plannedto be fixed, and the notes on :func:`solve_cubic` and :func:`solve_quartic` give-the rates and a workaround.+the rates and a workaround. :func:`multiply_deg_one_poly`, :func:`multiply_deg_two_one_poly` and+:func:`determinant_to_polynomial` are steps of Nister's five-point solver, which+:func:`~kornia.geometry.epipolar.find_essential` runs whole... autofunction:: solve_quadratic.. autofunction:: solve_cubicdiff --git a/docs/source/geometry.transform.rst b/docs/source/geometry.transform.rstindex 5bd4fa2b9..a205345bf 100644--- a/docs/source/geometry.transform.rst+++ b/docs/source/geometry.transform.rst@@ -6,11 +6,24 @@ kornia.geometry.transform.. currentmodule:: kornia.geometry.transform-The functions in this section perform various geometric transformations of 2D and 3D images.+The functions in this section warp, resize, crop, flip and register 2D images and 3D volumes. The warps take a+matrix that maps source pixels to destination pixels (:func:`warp_perspective`, :func:`warp_affine` and their 3D+forms), except :func:`homography_warp` and :func:`homography_warp3d`, which take a destinationāsource homography in+normalized :math:`[-1, 1]` coordinates by default. The 2D image transforms :func:`rotate`, :func:`translate`,+:func:`scale` and :func:`shear` take parameters instead of a matrix and :func:`affine` takes a matrix, all at the+image size; :func:`resize` and :func:`rescale` change the size. ``align_corners`` defaults differ between these+functions; see :doc:`Conventions & Pitfalls </get-started/conventions>`.Warp operators--------------+Warp images ``(B, C, H, W)`` or volumes ``(B, C, D, H, W)`` to the output size ``dsize`` by a sourceādestination+pixel homography (:func:`warp_perspective`, :func:`warp_perspective3d`) or affine matrix (:func:`warp_affine`,+:func:`warp_affine3d`), or by a destinationāsource homography in normalized coordinates (:func:`homography_warp`,+:func:`homography_warp3d`). Sample an image at per-pixel source coordinates with :func:`remap`, or warp it by+thin-plate-spline control points with :func:`warp_image_tps` (and points with :func:`warp_points_tps`).+:func:`warp_grid` and :func:`warp_grid3d` return the warped sampling grid without sampling.+.. autofunction:: warp_perspective.. autofunction:: warp_perspective3d.. autofunction:: warp_affine@@ -26,6 +39,14 @@ Warp operatorsImage 2D transforms-------------------+Rotate, shift, zoom or shear an image by parameters (:func:`rotate`, :func:`translate`, :func:`scale`,+:func:`shear`), or warp it by an affine matrix (:func:`affine`), at the input size; change the size with+:func:`resize`, :func:`rescale` or :func:`resize_to_be_divisible`. Mirror or turn it by reversing axes, without+interpolation (:func:`hflip`, :func:`vflip`, :func:`rot180`); deform it by a smoothed displacement field+(:func:`elastic_transform2d`); blur and resize it by one pyramid level (:func:`pyrdown`, :func:`pyrup`), or build a+Gaussian or Laplacian pyramid (:func:`build_pyramid`, :func:`build_laplacian_pyramid`, with the integer helpers+:func:`is_powerof_two` and :func:`find_next_powerof_two`).+.. autofunction:: affine.. autofunction:: rotate.. autofunction:: translate@@ -48,12 +69,22 @@ Image 2D transformsImage 3D transforms-------------------+Warp a volume by an affine matrix (:func:`affine3d`) or rotate it about its centre by an axis-angle vector+(:func:`rotate3d`), at the input size.+.. autofunction:: affine3d.. autofunction:: rotate3dMatrix transformations----------------------+Build the matrices the warps take: a homography from point pairs (:func:`get_perspective_transform`, and+:func:`get_perspective_transform3d` from box corners), an affine matrix from a rotation, a translation, a shear or+all of them (:func:`get_rotation_matrix2d`, :func:`get_translation_matrix2d`, :func:`get_shear_matrix2d`,+:func:`get_affine_matrix2d`, and in 3D :func:`get_projective_transform`, :func:`get_shear_matrix3d`,+:func:`get_affine_matrix3d`), the inverse of an affine matrix (:func:`invert_affine_transform`) and ``[R | t]``+(:func:`projection_from_Rt`). :func:`get_tps_transform` fits the thin-plate spline that :func:`warp_image_tps` takes.+.. autofunction:: get_perspective_transform.. autofunction:: get_perspective_transform3d.. autofunction:: get_projective_transform@@ -70,6 +101,12 @@ Matrix transformationsCrop operators--------------+Crop ``(B, C, H, W)`` images by integer slicing of axis-aligned boxes (:func:`crop_by_indices`), by resampling a+four-corner box to a fixed size (:func:`crop_and_resize`) or onto a second box whose size sets the output+(:func:`crop_by_boxes`), by a sourceādestination pixel matrix (:func:`crop_by_transform_mat`), or at the centre+(:func:`center_crop`, and its module form :class:`CenterCrop2D`). A box is ``(B, 4, 2)``: its ``(x, y)`` corners+clockwise from the top-left, as inclusive pixel positions.+.. autofunction:: crop_by_indices.. autofunction:: crop_by_boxes.. autofunction:: crop_by_transform_mat@@ -79,6 +116,11 @@ Crop operatorsCrop operators (3D)--------------------+Crop ``(B, C, D, H, W)`` volumes by ``(B, 8, 3)`` boxes, resampled to a fixed size (:func:`crop_and_resize3d`) or onto+a second box (:func:`crop_by_boxes3d`), by a sourceādestination voxel matrix (:func:`crop_by_transform_mat3d`), or at+the centre (:func:`center_crop3d`). Pass ``align_corners=True``: at ``False`` the 3D crops sample off the box+(`#4503 <https://github.com/kornia/kornia/issues/4503>`_).+.. autofunction:: crop_by_boxes3d.. autofunction:: crop_by_transform_mat3d.. autofunction:: center_crop3d@@ -87,6 +129,13 @@ Crop operators (3D)Modules-------+The ``torch.nn.Module`` classes. Module forms that store their arguments: the 2D transforms (:class:`Rotate`,+:class:`Translate`, :class:`Scale`, :class:`Shear`, :class:`Affine`), the resizes (:class:`Resize`, :class:`Rescale`),+the pyramid steps (:class:`PyrDown`, :class:`PyrUp`) and the centre crop (:class:`CenterCrop2D`); module forms that+take no arguments: the flips (:class:`Hflip`, :class:`Vflip`, :class:`Rot180`). :class:`ScalePyramid` builds+scale-space octaves, :class:`HomographyWarper` warps by a normalized homography, and :class:`BaseWarper` is the+interface for writing another warper.+.. autoclass:: Rotate.. autoclass:: Translate.. autoclass:: Scalediff --git a/kornia/geometry/__init__.py b/kornia/geometry/__init__.pyindex c054e12e0..cd2c60b3e 100644--- a/kornia/geometry/__init__.py+++ b/kornia/geometry/__init__.py@@ -15,9 +15,40 @@# limitations under the License.#-"""Kornia Geometry ā Geometric vision operations for Kornia.+r"""Geometric computer vision: warp images, model cameras, estimate two-view geometry, represent 3D.-This subpackage provides modules for transformations, camera models, and geometry processing.+The subpackages share kornia's conventions: images are ``(B, C, H, W)``, points and vectors sit in the last+dimension, pixel coordinates are ``(x, y)`` with ``x`` the column and pixel centres at integers, matrices act on+column vectors from the left, and camera extrinsics are world-to-camera, :math:`X_{cam} = R X + t`.+:doc:`/get-started/conventions` and :doc:`/get-started/camera-conventions` give the details. ``kornia.geometry``+re-exports the names of every subpackage below except ``boxes``, ``keypoints`` and ``vector`` (import those from+their modules, such as ``kornia.geometry.boxes.Boxes``), and of ``quaternion`` only ``Quaternion``.++Which subpackage to use:++- Warp, resize, crop, flip or register images, and build the matrices of those warps: ``transform``+ (:func:`~kornia.geometry.transform.warp_perspective`, :func:`~kornia.geometry.transform.warp_affine`,+ :func:`~kornia.geometry.transform.resize`).+- Convert between homogeneous and Euclidean points, pixel and normalized coordinates, rotation forms and pose+ conventions: ``conversions``.+- Apply a homogeneous ``(D + 1, D + 1)`` transformation to points; compose, invert or relate ``4 x 4``+ transformations; distances of points to points and lines: ``linalg``. Grids of pixel coordinates: ``grid``.+- Camera models, and the projection and unprojection of points: ``camera``+ (:class:`~kornia.geometry.camera.pinhole.PinholeCamera`); lens distortion and pose from 2D-3D correspondences:+ ``calibration``.+- 3D points and surface normals from depth maps, and images warped by depth: ``depth``.+- Two-view geometry, fundamental and essential matrices, relative pose and triangulation: ``epipolar``;+ homographies from points or line segments: ``homography``; any of these models robust to outliers: ``ransac``+ (:class:`~kornia.geometry.ransac.RANSAC`); the affine, null-vector and polynomial solvers they build on:+ ``solvers``.+- Rotations and rigid motions as objects that compose and invert: ``liegroup`` (:class:`~kornia.geometry.liegroup.So2`,+ :class:`~kornia.geometry.liegroup.So3`, :class:`~kornia.geometry.liegroup.Se2`,+ :class:`~kornia.geometry.liegroup.Se3`), ``quaternion`` (:class:`~kornia.geometry.quaternion.Quaternion`), and+ poses between named frames, ``pose`` (:class:`~kornia.geometry.pose.NamedPose`).+- Geometric primitives: 2D and 3D boxes, ``boxes`` (:class:`~kornia.geometry.boxes.Boxes`), with the tensor+ functions of ``bbox``; keypoints, ``keypoints``; lines, ``line``; planes, ``plane``; 2D and 3D vectors, ``vector``.+- Sub-pixel peaks, soft argmax and non-maxima suppression of response maps: ``subpix``.+- Point clouds to and from PLY files: ``pointcloud``."""from .bbox import *diff --git a/kornia/geometry/conversions.py b/kornia/geometry/conversions.pyindex 57f588ee3..c36943382 100644--- a/kornia/geometry/conversions.py+++ b/kornia/geometry/conversions.py@@ -15,6 +15,57 @@# limitations under the License.#+"""Convert between homogeneous points, normalized coordinates, rotations and poses.++Points and vectors sit in the last dimension, and matrices act on column vectors from the left. 2D pixel coordinates+are ``(x, y)``, with ``x`` the column and pixel centres at integers, while size arguments are ``(height, width)`` or+``(depth, height, width)``. The :math:`[-1, 1]` normalizations are corner-aligned, mapping the first and last pixel+centre of an axis to ``-1`` and ``1``; only :func:`normal_transform_pixel`, :func:`normalize_homography` and+:func:`denormalize_homography` also offer the half-pixel convention of ``align_corners=False``, and their 3D forms+do not (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). Quaternions are ``(w, x, y, z)``, real part first,+in the Hamilton convention. 3D rotations are active and right-handed: a positive angle about ``+z`` turns ``+x``+towards ``+y``. Axis-angle vectors and Euler angles are in radians, while :func:`angle_to_rotation_matrix` takes+degrees and turns ``+x`` towards ``-y``, counter-clockwise as displayed with ``y`` down. The pose functions take+exactly one batch dimension: ``(B, 3, 3)`` rotations with ``(B, 3, 1)`` translations, ``(B, 4, 4)`` matrices or+``(B, 4)`` quaternions.++Which function to use:++- Homogeneous points: :func:`convert_points_to_homogeneous` appends ``w = 1`` to ``(*, N, D)`` points and+ :func:`convert_points_from_homogeneous` divides by it. An affine matrix as a homography:+ :func:`convert_affinematrix_to_homography` for ``(B, 2, 3)``, :func:`convert_affinematrix_to_homography3d` for+ ``(B, 3, 4)``.+- Pixel coordinates to :math:`[-1, 1]` and back: :func:`normalize_pixel_coordinates` and+ :func:`denormalize_pixel_coordinates` for ``(*, 2)`` points, :func:`normalize_pixel_coordinates3d` and+ :func:`denormalize_pixel_coordinates3d` for ``(*, 3)`` points in ``(d, x, y)`` order. The same maps as matrices:+ :func:`normal_transform_pixel` ``(1, 3, 3)`` on ``(x, y, 1)`` and :func:`normal_transform_pixel3d` ``(1, 4, 4)`` on+ ``(x, y, z, 1)``.+- A pixel homography in :math:`[-1, 1]` coordinates and back: :func:`normalize_homography` and+ :func:`denormalize_homography` for ``(B, 3, 3)``, :func:`normalize_homography3d` and+ :func:`denormalize_homography3d` for ``(B, 4, 4)``.+- Pixels to normalized camera coordinates with intrinsics ``K``, and back: :func:`normalize_points_with_intrinsics`,+ :func:`denormalize_points_with_intrinsics`.+- A 3D rotation from one form to another, axis-angle ``(*, 3)``, quaternion ``(*, 4)`` or matrix ``(*, 3, 3)``:+ :func:`axis_angle_to_rotation_matrix`, :func:`axis_angle_to_quaternion`, :func:`quaternion_to_rotation_matrix`,+ :func:`quaternion_to_axis_angle`, :func:`rotation_matrix_to_quaternion`, :func:`rotation_matrix_to_axis_angle`.+ The deprecated names ``angle_axis_to_rotation_matrix``, ``angle_axis_to_quaternion``, ``quaternion_to_angle_axis``+ and ``rotation_matrix_to_angle_axis`` forward to them.+- Quaternion helpers: unit length, :func:`normalize_quaternion`; the exponential and logarithm maps, whose 3-vector+ is half the axis-angle vector, :func:`quaternion_log_to_exp` and :func:`quaternion_exp_to_log`; roll, pitch and+ yaw, with the quaternion as four separate tensors, :func:`quaternion_from_euler` and :func:`euler_from_quaternion`.+- A 2D rotation matrix from an angle in degrees: :func:`angle_to_rotation_matrix`. The cross-product matrix of a+ 3-vector: :func:`vector_to_skew_symmetric_matrix`.+- Poses: pack and split ``(B, 4, 4)`` matrices with :func:`Rt_to_matrix4x4` and :func:`matrix4x4_to_Rt`; invert a+ rigid ``(R, t)`` pose with :func:`camtoworld_to_worldtocam_Rt` or :func:`worldtocam_to_camtoworld_Rt`; switch a+ camera-to-world pose between OpenGL and OpenCV camera axes with :func:`camtoworld_graphics_to_vision_4x4`,+ :func:`camtoworld_graphics_to_vision_Rt`, :func:`camtoworld_vision_to_graphics_4x4` or+ :func:`camtoworld_vision_to_graphics_Rt`; convert quaternion poses between ARKit and Colmap with+ :func:`ARKitQTVecs_to_ColmapQTVecs` and :func:`ColmapQTVecs_to_ARKitQTVecs`.+- Angles and polar coordinates: :func:`deg2rad`, :func:`rad2deg`, :func:`cart2pol`, :func:`pol2cart`.+- Rotations and poses as objects that compose and invert: :class:`~kornia.geometry.quaternion.Quaternion`,+ :class:`~kornia.geometry.liegroup.So3` and :class:`~kornia.geometry.liegroup.Se3`.+"""+from __future__ import annotationsimport math@@ -82,17 +133,20 @@ __all__ = [def rad2deg(tensor: torch.Tensor) -> torch.Tensor:- r"""Convert angles from radians to degrees.+ r"""Convert angles from radians to degrees, elementwise.- Convention:- - the input is in **radians** and the output in **degrees**; the- conversion is elementwise and preserves shape, device and float dtype+ ``degrees = rad2deg(radians)`` multiplies a tensor of any shape by ``180 / pi`` and returns the angles in degrees+ with the same shape, device and floating dtype; an integer tensor gives the default floating dtype. For the+ reverse, use :func:`deg2rad`.Args:- tensor: torch.Tensor of arbitrary shape.+ tensor: angles in radians, of any shape.Returns:- torch.Tensor with same shape as input.+ the angles in degrees, with the shape of ``tensor``.++ See Also:+ - :func:`deg2rad`: the reverse conversion, degrees to radians.Example:>>> input = torch.tensor(3.1415926535)@@ -107,18 +161,22 @@ def rad2deg(tensor: torch.Tensor) -> torch.Tensor:def deg2rad(tensor: torch.Tensor) -> torch.Tensor:- r"""Convert angles from degrees to radians.+ r"""Convert angles from degrees to radians, elementwise.- Convention:- - the input is in **degrees** and the output in **radians**; it- performs the opposite conversion to- :func:`~kornia.geometry.conversions.rad2deg`+ ``radians = deg2rad(degrees)`` multiplies a tensor of any shape by ``pi / 180`` and returns the angles in radians+ with the same shape, device and floating dtype; an integer tensor gives the default floating dtype. The axis-angle,+ Euler and polar functions of this module take radians; :func:`angle_to_rotation_matrix` takes degrees. For the+ reverse, use :func:`rad2deg`.Args:- tensor: torch.Tensor of arbitrary shape.+ tensor: angles in degrees, of any shape.Returns:- tensor with same shape as input.+ the angles in radians, with the shape of ``tensor``.++ See Also:+ - :func:`rad2deg`: the reverse conversion, radians to degrees.+ - :func:`angle_to_rotation_matrix`: the 2D rotation matrix of an angle given in degrees.Examples:>>> input = torch.tensor(180.)@@ -133,23 +191,31 @@ def deg2rad(tensor: torch.Tensor) -> torch.Tensor:def pol2cart(rho: torch.Tensor, phi: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert polar coordinates to cartesian coordinates.+ r"""Convert polar coordinates (rho, phi), phi in radians, to cartesian coordinates (x, y).++ ``x, y = pol2cart(rho, phi)`` takes radii and angles in radians and returns two tensors, ``x = rho * cos(phi)`` and+ ``y = rho * sin(phi)``, in the broadcast shape of ``rho`` and ``phi``. It undoes :func:`cart2pol`:+ ``pol2cart(*cart2pol(x, y))`` returns ``(x, y)`` to rounding as long as ``x ** 2 + y ** 2`` is above the ``eps``+ of :func:`cart2pol` and neither overflows nor underflows in the working dtype. Inside the ``eps`` neighbourhood of+ the origin :func:`cart2pol` rounds the radius, so in ``float32`` ``(1e-5, 2e-5)`` comes back as+ ``(1.0023e-5, 2.0045e-5)`` and ``(3e-8, 4e-8)`` as ``(0, 0)``+ (`#6440 <https://github.com/kornia/kornia/issues/6440>`_).Convention:- - the arguments are ``(rho, phi)`` and the return is the tuple- ``(x, y)``, with ``x = rho * cos(phi)`` and ``y = rho * sin(phi)``:- ``rho = 5``, ``phi = atan2(4, 3)`` gives ``(3., 4.)``- - ``phi`` is in **radians** (``rho = 2``, ``phi = pi / 6`` gives- ``(1.7321, 1.)``) and follows the angle convention documented on- :func:`~kornia.geometry.conversions.cart2pol`+ - ``rho = 5``, ``phi = atan2(4, 3)`` gives ``(3., 4.)``, and ``rho = 2``, ``phi = pi / 6`` gives+ ``(1.7321, 1.)``+ - ``phi`` follows the angle convention documented on :func:`cart2pol`Args:- rho: torch.Tensor of arbitrary shape.- phi: torch.Tensor of same arbitrary shape.+ rho: the radii, of any shape that broadcasts with ``phi``.+ phi: the angles in radians.Returns:- - x: torch.Tensor with same shape as input.- - y: torch.Tensor with same shape as input.+ - x: the first coordinate, in the broadcast shape of ``rho`` and ``phi``.+ - y: the second coordinate, in the same shape.++ See Also:+ - :func:`cart2pol`: the reverse conversion, cartesian to polar.Example:>>> rho = torch.rand(1, 3, 3)@@ -166,11 +232,33 @@ def pol2cart(rho: torch.Tensor, phi: torch.Tensor) -> tuple[torch.Tensor, torch.def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert cartesian coordinates to polar coordinates.+ r"""Convert cartesian coordinates (x, y) to polar coordinates (rho, phi), phi in radians.++ ``rho, phi = cart2pol(x, y)`` returns two tensors in the broadcast shape of ``x`` and ``y``: the radius+ ``sqrt(x ** 2 + y ** 2)`` and the angle ``atan2(y, x)`` in radians, in :math:`[-\pi, \pi]`. The radius is+ computed from the unscaled ``x ** 2 + y ** 2``, so it is ``nan`` once that overflows+ (`#6414 <https://github.com/kornia/kornia/issues/6414>`_). ``eps`` keeps the gradient of ``rho`` finite at the+ origin, but it also rounds every radius with ``x ** 2 + y ** 2 <= eps``, as the convention below details+ (`#6440 <https://github.com/kornia/kornia/issues/6440>`_). For the reverse, use :func:`pol2cart`.Convention:- - the arguments are ``(x, y)`` and the return is the tuple- ``(rho, phi)``: ``(3., 4.)`` gives ``(5., 0.9273)``+ - ``(3., 4.)`` gives ``(5., 0.9273)``+ - ``x ** 2 + y ** 2`` overflows from a radius of about ``256`` in+ ``float16`` (``(300., 150.)`` gives ``rho = nan``), ``1.8e19`` in+ ``float32`` and ``bfloat16`` and ``1.3e154`` in ``float64``, and+ ``rho`` is then ``nan``, not ``inf``+ (`#6414 <https://github.com/kornia/kornia/issues/6414>`_)+ - where ``x ** 2 + y ** 2 <= eps`` (radii up to about ``1e-4`` at the+ default ``eps``), ``rho`` is computed as ``(rho + 1) - 1``, which+ rounds it to a multiple of the spacing of the dtype at ``1``+ (`#6440 <https://github.com/kornia/kornia/issues/6440>`_): in+ ``float32`` ``(1e-5, 0.)`` gives ``1.0014e-5`` and every radius below+ about ``6e-8``, ``(1e-20, 0.)`` included, gives ``0``; in ``bfloat16``+ every radius up to ``1e-4`` gives ``0``; in ``float64`` ``(1e-14, 0.)``+ gives ``9.992e-15``. In ``float16`` no nonzero sum is that small, so+ only a sum of ``0`` takes that branch; there a sum that underflows+ makes ``rho`` inexact or ``0`` instead: ``(2e-4, 0.)`` gives+ ``2.44e-4``- ``phi`` is ``atan2(y, x)`` in **radians**, so it lies in:math:`[-\pi, \pi]`, is ``0`` on the ``+x`` axis and grows toward``+y`` (``x = 0``, ``y = 1`` gives ``phi = 1.5708``). Both endpoints@@ -185,24 +273,23 @@ def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[tor``phi = -170`` degrees rotated by ``theta = 30`` returns``phi = +160``, not ``-200``. At the origin ``phi`` carries nodirection and the relation does not apply- - ``rho`` is the Euclidean radius ``sqrt(x ** 2 + y ** 2)``.- ``eps`` only protects the gradient near the origin and does not- perturb the returned radius-- .. note::- ``eps`` defines the neighborhood around the origin where the gradient- is stabilized. It does not change the forward value of ``rho``.Args:- x: torch.Tensor of arbitrary shape.- y: torch.Tensor of same arbitrary shape.- eps: squared-radius threshold used to stabilize the gradient near the- origin. It does not perturb the forward value of ``rho``. The+ x: the first coordinates, of any shape that broadcasts with ``y``.+ y: the second coordinates.+ eps: squared-radius threshold: at or below it the gradient of ``rho``+ is zero, which avoids the ``0 / 0`` of the exact gradient at the+ origin, and ``rho`` is rounded as described above+ (`#6440 <https://github.com/kornia/kornia/issues/6440>`_). Thedefault ``1e-8`` underflows in ``float16``.Returns:- - rho: torch.Tensor with same shape as input.- - phi: torch.Tensor with same shape as input.+ - rho: the radius, in the broadcast shape of ``x`` and ``y``.+ - phi: the angle in radians, in the same shape.++ See Also:+ - :func:`pol2cart`: the reverse conversion, polar to cartesian.+ - :func:`angle_to_rotation_matrix`: the 2D rotation by an angle in degrees, which turns ``phi`` the other way.Example:>>> x = torch.rand(1, 3, 3)@@ -222,11 +309,14 @@ def cart2pol(x: torch.Tensor, y: torch.Tensor, eps: float = 1.0e-8) -> tuple[tordef convert_points_from_homogeneous(points: torch.Tensor, eps: float = 1e-8) -> torch.Tensor:- r"""Convert points from homogeneous to Euclidean space.+ r"""Convert homogeneous points ``(*, N, D)`` to Euclidean ``(*, N, D - 1)``.++ ``points = convert_points_from_homogeneous(points_h)`` takes points ``(*, N, D)`` whose **last** coordinate is the+ homogeneous ``w``, divides the others by it and returns ``(*, N, D - 1)``. A point with ``abs(w) <= eps`` comes+ back undivided rather than as ``inf``, following OpenCV. For the reverse, use :func:`convert_points_to_homogeneous`;+ to apply a homography to Euclidean points in one call, :func:`~kornia.geometry.linalg.transform_points`.Convention:- - the **last** component of the last dimension is the homogeneous- coordinate ``w`` and is dropped from the output- requires rank :math:`\geq 2`: a bare ``(D,)`` point raises``ValueError``- when ``abs(w) > eps`` the remaining components are divided by exactly@@ -246,6 +336,11 @@ def convert_points_from_homogeneous(points: torch.Tensor, eps: float = 1e-8) ->Returns:the points in Euclidean space ``(*, N, D-1)``.+ See Also:+ - :func:`convert_points_to_homogeneous`: the reverse, appending ``w = 1``.+ - :func:`~kornia.geometry.linalg.transform_points`: apply a ``(B, D + 1, D + 1)`` matrix to Euclidean points,+ with this division included.+Examples:>>> input = torch.tensor([[0., 0., 1.]])>>> convert_points_from_homogeneous(input)@@ -272,16 +367,15 @@ def convert_points_from_homogeneous(points: torch.Tensor, eps: float = 1e-8) ->def convert_points_to_homogeneous(points: torch.Tensor) -> torch.Tensor:- r"""Convert points from Euclidean to homogeneous space.+ r"""Convert Euclidean points ``(*, N, D)`` to homogeneous ``(*, N, D + 1)`` by appending a 1.++ ``points_h = convert_points_to_homogeneous(points)`` appends the coordinate ``w = 1`` as the **last** component of+ each point ``(*, N, D)``, where :func:`convert_points_from_homogeneous` expects it, and returns ``(*, N, D + 1)`` in+ the input dtype, integer dtypes included.Convention:- - appends the constant ``1`` as the **last** component of the last- dimension, which is where- :func:`~kornia.geometry.conversions.convert_points_from_homogeneous`- expects it- requires rank :math:`\geq 2`: a bare ``(D,)`` point raises``ValueError``- - the input dtype is preserved, integer dtypes includedArgs:points: the points to be transformed with shape ``(*, N, D)``.@@ -289,6 +383,9 @@ def convert_points_to_homogeneous(points: torch.Tensor) -> torch.Tensor:Returns:the points in homogeneous coordinates ``(*, N, D+1)``.+ See Also:+ - :func:`convert_points_from_homogeneous`: the reverse, dividing by ``w`` and dropping it.+Examples:>>> input = torch.tensor([[0., 0.]])>>> convert_points_to_homogeneous(input)@@ -312,11 +409,14 @@ def _convert_affinematrix_to_homography_impl(A: torch.Tensor) -> torch.Tensor:def convert_affinematrix_to_homography(A: torch.Tensor) -> torch.Tensor:- r"""Convert batch of affine matrices.+ r"""Extend 2D affine matrices ``(B, 2, 3)`` to ``(B, 3, 3)`` homographies.++ ``H = convert_affinematrix_to_homography(A)`` copies each ``(B, 2, 3)`` affine matrix verbatim into the top of a+ ``(B, 3, 3)`` matrix whose last row is ``[0, 0, 1]``, so that it composes with homographies by matrix product and+ inverts with :func:`torch.linalg.inv`; the input is not modified. For ``(B, 3, 4)`` 3D affine matrices, use+ :func:`convert_affinematrix_to_homography3d`.Convention:- - appends the row ``[0, 0, 1]`` at the bottom and copies the- :math:`2 \times 3` block verbatim; the input tensor is not modified- the rank is enforced to be exactly 3, so batching is **mandatory**: anunbatched ``(2, 3)`` matrix raises ``ValueError``@@ -326,6 +426,11 @@ def convert_affinematrix_to_homography(A: torch.Tensor) -> torch.Tensor:Returns:the homography matrix with shape of ``(B,3,3)``.+ See Also:+ - :func:`convert_affinematrix_to_homography3d`: the 3D form, ``(B, 3, 4)`` to ``(B, 4, 4)``.+ - :func:`~kornia.geometry.transform.invert_affine_transform`: the inverse of a ``(B, 2, 3)`` matrix, returned+ as ``(B, 2, 3)``.+Examples:>>> A = torch.tensor([[[1., 0., 0.],... [0., 1., 0.]]])@@ -345,13 +450,16 @@ def convert_affinematrix_to_homography(A: torch.Tensor) -> torch.Tensor:def convert_affinematrix_to_homography3d(A: torch.Tensor) -> torch.Tensor:- r"""Convert batch of 3d affine matrices.+ r"""Extend 3D affine matrices ``(B, 3, 4)`` to ``(B, 4, 4)`` homographies.++ ``H = convert_affinematrix_to_homography3d(A)`` copies each ``(B, 3, 4)`` affine matrix verbatim into the top of a+ ``(B, 4, 4)`` matrix whose last row is ``[0, 0, 0, 1]``; the input is not modified. For a rotation and a+ translation given separately, use :func:`Rt_to_matrix4x4`; for ``(B, 2, 3)`` 2D matrices,+ :func:`convert_affinematrix_to_homography`.Convention:- - same as- :func:`~kornia.geometry.conversions.convert_affinematrix_to_homography`,- except that ``A`` is ``(B, 3, 4)`` and the appended bottom row is- ``[0, 0, 0, 1]``+ - the rank is enforced to be exactly 3, as in the 2D function: an+ unbatched ``(3, 4)`` matrix raises ``ValueError``Args:A: the affine matrix with shape ``(B,3,4)``.@@ -359,6 +467,11 @@ def convert_affinematrix_to_homography3d(A: torch.Tensor) -> torch.Tensor:Returns:the homography matrix with shape of ``(B,4,4)``.+ See Also:+ - :func:`convert_affinematrix_to_homography`: the 2D form, ``(B, 2, 3)`` to ``(B, 3, 3)``.+ - :func:`Rt_to_matrix4x4`: the same ``(B, 4, 4)`` matrix from a ``(B, 3, 3)`` rotation and a ``(B, 3, 1)``+ translation.+Examples:>>> A = torch.tensor([[[1., 0., 0., 0.],... [0., 1., 0., 0.],@@ -380,7 +493,13 @@ def convert_affinematrix_to_homography3d(A: torch.Tensor) -> torch.Tensor:def axis_angle_to_rotation_matrix(axis_angle: torch.Tensor) -> torch.Tensor:- r"""Convert 3d vector of axis-angle rotation to 3x3 rotation matrix.+ r"""Convert axis-angle vectors ``(*, 3)`` in radians to rotation matrices ``(*, 3, 3)``.++ ``R = axis_angle_to_rotation_matrix(axis_angle)`` takes rotation vectors ``(*, 3)``, each the unit axis scaled by+ the angle in radians, and returns the rotation matrices ``(*, 3, 3)`` by Rodrigues' formula, with any number of+ leading dimensions. The matrices act on column vectors from the left and turn by the right-hand rule. For the+ quaternion of the same rotation, use :func:`axis_angle_to_quaternion`; for the reverse,+ :func:`rotation_matrix_to_axis_angle`.Convention:- any number of leading batch dimensions is accepted: ``(3,)``@@ -411,6 +530,12 @@ def axis_angle_to_rotation_matrix(axis_angle: torch.Tensor) -> torch.Tensor:Returns:tensor of rotation matrices of shape ``(*, 3, 3)``.+ See Also:+ - :func:`rotation_matrix_to_axis_angle`: the reverse conversion, matrix to axis-angle.+ - :func:`axis_angle_to_quaternion`: the ``(w, x, y, z)`` quaternion of the same rotation.+ - :func:`quaternion_to_rotation_matrix`: the same matrix from a quaternion.+ - :func:`angle_to_rotation_matrix`: a 2D ``(*, 2, 2)`` rotation from an angle in degrees.+Example:>>> input = torch.tensor([[0., 0., 0.]])>>> axis_angle_to_rotation_matrix(input) # doctest: +ELLIPSIS@@ -509,26 +634,36 @@ def axis_angle_to_rotation_matrix(axis_angle: torch.Tensor) -> torch.Tensor:@deprecated(replace_with="axis_angle_to_rotation_matrix", version="0.7.0")-def angle_axis_to_rotation_matrix(axis_angle: torch.Tensor) -> torch.Tensor: # noqa: D103+def angle_axis_to_rotation_matrix(axis_angle: torch.Tensor) -> torch.Tensor:+ """Deprecated: use :func:`~kornia.geometry.conversions.axis_angle_to_rotation_matrix`.++ Each call emits a ``DeprecationWarning`` (deprecated since 0.7.0) and returns what the new name returns.+ """return axis_angle_to_rotation_matrix(axis_angle)def rotation_matrix_to_axis_angle(rotation_matrix: torch.Tensor, check_rotation: bool = False) -> torch.Tensor:- r"""Convert 3x3 rotation matrix to Rodrigues vector in radians.+ r"""Convert rotation matrices ``(*, 3, 3)`` to axis-angle vectors ``(*, 3)`` in radians.++ ``axis_angle = rotation_matrix_to_axis_angle(R)`` returns, for each rotation matrix ``(*, 3, 3)``, the Rodrigues+ vector ``(*, 3)``: the unit axis scaled by the angle in radians, at most ``pi``, the form+ :func:`axis_angle_to_rotation_matrix` takes back. It goes through :func:`rotation_matrix_to_quaternion` and+ :func:`quaternion_to_axis_angle`. The input is not checked to be a rotation unless ``check_rotation=True``. For the+ ``(w, x, y, z)`` quaternion instead, use :func:`rotation_matrix_to_quaternion`.Convention:- any number of leading batch dimensions is accepted: ``(3, 3)``gives ``(3,)`` and ``(2, 5, 3, 3)`` gives ``(2, 5, 3)``- - the output is the rotation axis scaled by the angle in **radians**,- the parametrization+ - the output is the parametrization:func:`~kornia.geometry.conversions.axis_angle_to_rotation_matrix`consumes, at every rank: the ``(3,)`` and ``(2, 5, 3)``results above can be fed straight back- - the round trip through- :func:`~kornia.geometry.conversions.axis_angle_to_rotation_matrix`- is accurate to roundoff, except that the low-angle Taylor branch- (``theta <= 1e-3``) leaves a truncation error of up to about- ``2e-10``+ - ``rotation_matrix_to_axis_angle(axis_angle_to_rotation_matrix(v))``+ returns ``v`` to roundoff for angles below ``pi``, the low-angle+ Taylor branch (``theta <= 1e-3``) included. A longer ``v`` comes+ back as the equivalent vector of angle at most ``pi``:+ ``[0., 0., 4.]`` returns ``[0., 0., -2.2832]``, a turn by+ ``2 * pi - 4`` about ``-z``- by default the input is **not** checked for being a rotation matrix:``zeros(3, 3)`` returns ``[0., 0., 3.1416]`` and ``2 * eye(3)``returns ``[0., 0., 0.]``. Pass ``check_rotation=True`` to reject@@ -551,6 +686,11 @@ def rotation_matrix_to_axis_angle(rotation_matrix: torch.Tensor, check_rotation:Returns:Rodrigues vector transformation of shape ``(*, 3)``.+ See Also:+ - :func:`axis_angle_to_rotation_matrix`: the reverse conversion, axis-angle to matrix.+ - :func:`rotation_matrix_to_quaternion`: the ``(w, x, y, z)`` quaternion of the same matrix.+ - :func:`quaternion_to_axis_angle`: the same vector from a quaternion.+Example:>>> input = torch.tensor([[1., 0., 0.],... [0., 1., 0.],@@ -579,21 +719,29 @@ def rotation_matrix_to_axis_angle(rotation_matrix: torch.Tensor, check_rotation:@deprecated(replace_with="rotation_matrix_to_axis_angle", version="0.7.0")-def rotation_matrix_to_angle_axis(rotation_matrix: torch.Tensor) -> torch.Tensor: # noqa: D103+def rotation_matrix_to_angle_axis(rotation_matrix: torch.Tensor) -> torch.Tensor:+ """Deprecated: use :func:`~kornia.geometry.conversions.rotation_matrix_to_axis_angle`.++ Each call emits a ``DeprecationWarning`` (deprecated since 0.7.0) and returns what the new name returns with its+ default ``check_rotation=False``: this name takes no ``check_rotation`` argument, so its input is never checked.+ """return rotation_matrix_to_axis_angle(rotation_matrix)def rotation_matrix_to_quaternion(rotation_matrix: torch.Tensor, eps: float = 1.0e-8, check_rotation: bool = False) -> torch.Tensor:- r"""Convert 3x3 rotation matrix to 4d quaternion vector.+ r"""Convert rotation matrices ``(*, 3, 3)`` to quaternions ``(*, 4)`` in ``(w, x, y, z)`` order.- The quaternion vector has components in (w, x, y, z) format.+ ``q = rotation_matrix_to_quaternion(R)`` returns, for each rotation matrix ``(*, 3, 3)``, the unit quaternion+ ``(*, 4)`` with the real part first, with any number of leading dimensions. Of the two quaternions ``q`` and ``-q``+ of a rotation it returns either, not always the one with ``w >= 0``. The input is not checked to be a rotation+ unless ``check_rotation=True``. For the reverse, use :func:`quaternion_to_rotation_matrix`; for the axis-angle+ vector, :func:`rotation_matrix_to_axis_angle`.Convention:- - the returned components are ``(w, x, y, z)``, real part first ā the- layout, and its silent misreading, are spelled out on- :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`+ - the ``(w, x, y, z)`` layout, and its silent misreading, are spelled+ out on :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`- the sign is **not** canonicalised to ``w >= 0``. For``trace(R) > 0`` the returned ``w`` is non-negative, but for``trace(R) <= 0`` it is the dominant of ``x, y, z`` that is forced@@ -622,6 +770,13 @@ def rotation_matrix_to_quaternion(Return:the rotation in quaternion with shape ``(*, 4)``.+ See Also:+ - :func:`quaternion_to_rotation_matrix`: the reverse conversion, quaternion to matrix.+ - :func:`rotation_matrix_to_axis_angle`: the axis-angle vector of the same matrix.+ - :func:`axis_angle_to_quaternion`: the quaternion of an axis-angle vector instead.+ - :meth:`~kornia.geometry.quaternion.Quaternion.from_matrix`: the same quaternion as a+ :class:`~kornia.geometry.quaternion.Quaternion` object.+Example:>>> input = torch.tensor([[1., 0., 0.],... [0., 1., 0.],@@ -702,9 +857,12 @@ def rotation_matrix_to_quaternion(def normalize_quaternion(quaternion: torch.Tensor, eps: float = 1.0e-12) -> torch.Tensor:- r"""Normalize a quaternion.+ r"""Scale quaternions ``(*, 4)`` to unit length, keeping their sign and component order.- The quaternion should be in (x, y, z, w) or (w, x, y, z) format.+ ``q_unit = normalize_quaternion(q)`` divides each quaternion ``(*, 4)`` by its L2 norm and returns ``(*, 4)``. It+ reads no component order, so the quaternion can be in ``(x, y, z, w)`` or ``(w, x, y, z)`` format. The norm is+ floored at ``eps``, so with a positive ``eps`` the zero quaternion returns zeros rather than ``nan``. The+ conversions of this module that take a quaternion accept a non-unit one, so they need no call here first.Convention:- it is an L2 normalisation of the **last** axis and nothing else, which@@ -731,6 +889,11 @@ def normalize_quaternion(quaternion: torch.Tensor, eps: float = 1.0e-12) -> torcReturn:the normalized quaternion of shape ``(*, 4)``.+ See Also:+ - :func:`quaternion_to_rotation_matrix`: normalizes with this function before building the matrix.+ - :meth:`~kornia.geometry.quaternion.Quaternion.normalize`: the same normalization of a+ :class:`~kornia.geometry.quaternion.Quaternion` object.+Example:>>> quaternion = torch.tensor((1., 0., 1., 0.))>>> normalize_quaternion(quaternion)@@ -780,14 +943,16 @@ def _normalize_last_dim(x: torch.Tensor, eps: float) -> torch.Tensor:def quaternion_to_rotation_matrix(quaternion: torch.Tensor) -> torch.Tensor:- r"""Convert a quaternion to a rotation matrix.+ r"""Convert ``(w, x, y, z)`` quaternions ``(*, 4)`` to rotation matrices ``(*, 3, 3)``.- The quaternion should be in (w, x, y, z) format.+ ``R = quaternion_to_rotation_matrix(q)`` takes quaternions ``(*, 4)`` with the **real part first**, in the Hamilton+ convention, and returns the rotation matrices ``(*, 3, 3)``, which act on column vectors from the left. The input+ is normalized first, so it need not be unit, and ``q`` and ``-q`` give the same matrix. A quaternion in+ ``(x, y, z, w)`` order is not detected and returns a valid-looking wrong rotation. For the reverse, use+ :func:`rotation_matrix_to_quaternion`; from an axis-angle vector, :func:`axis_angle_to_rotation_matrix`.Convention:- - the quaternion is ``(w, x, y, z)``, **real part first**, shape- ``(*, 4)`` in and ``(*, 3, 3)`` out. ``[1., 0., 0., 0.]``- gives the identity and ``[0., 0., 0., 1.]`` gives+ - ``[1., 0., 0., 0.]`` gives the identity and ``[0., 0., 0., 1.]`` gives``[[-1., 0., 0.], [0., -1., 0.], [0., 0., 1.]]``, a half turn about``+z`` ā so a caller passing ``(x, y, z, w)`` gets a valid-lookingrotation matrix and no error@@ -818,6 +983,14 @@ def quaternion_to_rotation_matrix(quaternion: torch.Tensor) -> torch.Tensor:Return:the rotation matrix of shape ``(*, 3, 3)``.+ See Also:+ - :func:`rotation_matrix_to_quaternion`: the reverse conversion, matrix to quaternion.+ - :func:`quaternion_to_axis_angle`: the axis-angle vector of the same quaternion.+ - :func:`axis_angle_to_rotation_matrix`: the same matrix from an axis-angle vector.+ - :func:`normalize_quaternion`: the normalization applied to the input first.+ - :meth:`~kornia.geometry.quaternion.Quaternion.matrix`: the same matrix of a+ :class:`~kornia.geometry.quaternion.Quaternion` object.+Example:>>> quaternion = torch.tensor((0., 0., 0., 1.))>>> quaternion_to_rotation_matrix(quaternion)@@ -877,17 +1050,19 @@ def quaternion_to_rotation_matrix(quaternion: torch.Tensor) -> torch.Tensor:def quaternion_to_axis_angle(quaternion: torch.Tensor) -> torch.Tensor:- """Convert quaternion vector to axis angle of rotation in radians.+ """Convert ``(w, x, y, z)`` quaternions ``(*, 4)`` to axis-angle vectors ``(*, 3)`` in radians.- The quaternion should be in (w, x, y, z) format.+ ``axis_angle = quaternion_to_axis_angle(q)`` returns, for each quaternion ``(*, 4)`` with the real part first, the+ rotation vector ``(*, 3)``: the unit axis scaled by the angle in radians, at most ``pi``. The input need not be+ unit, and ``q`` and ``-q`` give the same vector unless ``w = 0``. For the reverse, use+ :func:`axis_angle_to_quaternion`; for the logarithm map, which is half this vector for ``w >= 0`` and tells ``q``+ from ``-q``, :func:`quaternion_exp_to_log`.Adapted from ceres C++ library: ceres-solver/include/ceres/rotation.hConvention:- - the input is ``(w, x, y, z)``, real part first (see- :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`),- and the output is the rotation axis scaled by the angle in- **radians**+ - the ``(w, x, y, z)`` layout is spelled out on+ :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`- for ``w != 0`` the double cover is collapsed: ``q`` and ``-q`` returnthe same vector, the representative with ``|theta| <= pi``- at exactly ``w = 0`` ā a half turn, and ``0.`` is exactly@@ -917,6 +1092,13 @@ def quaternion_to_axis_angle(quaternion: torch.Tensor) -> torch.Tensor:- Input: ``(*, 4)`` where `*` means, any number of dimensions- Output: ``(*, 3)``+ See Also:+ - :func:`axis_angle_to_quaternion`: the reverse conversion, axis-angle to quaternion.+ - :func:`quaternion_to_rotation_matrix`: the rotation matrix of the same quaternion.+ - :func:`quaternion_exp_to_log`: the logarithm map, half this vector for ``w >= 0``; it tells ``q`` from+ ``-q``.+ - :func:`rotation_matrix_to_axis_angle`: the same vector from a rotation matrix.+Example:>>> quaternion = torch.tensor((1., 0., 0., 0.))>>> quaternion_to_axis_angle(quaternion)@@ -997,18 +1179,26 @@ def quaternion_to_axis_angle(quaternion: torch.Tensor) -> torch.Tensor:@deprecated(replace_with="quaternion_to_axis_angle", version="0.7.0")-def quaternion_to_angle_axis(quaternion: torch.Tensor) -> torch.Tensor: # noqa: D103+def quaternion_to_angle_axis(quaternion: torch.Tensor) -> torch.Tensor:+ """Deprecated: use :func:`~kornia.geometry.conversions.quaternion_to_axis_angle`.++ Each call emits a ``DeprecationWarning`` (deprecated since 0.7.0) and returns what the new name returns.+ """return quaternion_to_axis_angle(quaternion)def quaternion_log_to_exp(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torch.Tensor:- r"""Apply exponential map to log quaternion.+ r"""Map log 3-vectors ``(*, 3)`` to ``(w, x, y, z)`` quaternions ``(*, 4)``, the exponential map.++ ``q = quaternion_log_to_exp(v)`` takes 3-vectors ``(*, 3)``, the logarithms of unit quaternions, and returns the+ quaternions ``(cos(||v||), sin(||v||) v / ||v||)`` of shape ``(*, 4)``, real part first. Despite its name, the+ argument ``quaternion`` is the 3-vector, which is **half** the axis-angle vector of the rotation; for the+ quaternion of an axis-angle vector, use :func:`axis_angle_to_quaternion`. For the reverse, use+ :func:`quaternion_exp_to_log`.Convention:- - the **input** is a 3-vector of shape ``(*, 3)``, not a- quaternion; the **output** is the quaternion of shape ``(*, 4)``- in ``(w, x, y, z)`` order (see- :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`)+ - the ``(w, x, y, z)`` layout is spelled out on+ :func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`- the map is ``w = cos(||v||)`` with vector part``sin(||v||) * v / ||v||``, and every finite ``v`` returns a unitquaternion up to the working dtype's rounding: ``||v||`` is computed@@ -1036,6 +1226,10 @@ def quaternion_log_to_exp(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torcReturn:the quaternion exponential map of shape ``(*, 4)``.+ See Also:+ - :func:`quaternion_exp_to_log`: the reverse, the logarithm map.+ - :func:`axis_angle_to_quaternion`: the same quaternion from the full axis-angle vector ``2 * v``.+Example:>>> quaternion = torch.tensor((0., 0., 0.))>>> quaternion_log_to_exp(quaternion, eps=torch.finfo(quaternion.dtype).eps)@@ -1072,33 +1266,37 @@ def quaternion_log_to_exp(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torcdef quaternion_exp_to_log(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torch.Tensor:- r"""Apply the log map to a quaternion.+ r"""Map ``(w, x, y, z)`` quaternions ``(*, 4)`` to log 3-vectors ``(*, 3)``, the logarithm map.- The quaternion should be in (w, x, y, z) format.+ ``v = quaternion_exp_to_log(q)`` takes quaternions ``(*, 4)`` with the real part first and returns the 3-vectors+ ``atan2(||u||, w) u / max(||u||, eps)`` of shape ``(*, 3)``, with ``u`` the vector part. While ``||u||`` is at+ least ``eps``, the input need not be unit and :func:`quaternion_log_to_exp` maps the result back to the unit+ quaternion of ``q``; below ``eps`` the log shrinks by ``||u|| / eps``, so ``(-1, 0, 0, 0)`` comes back as+ ``(1, 0, 0, 0)``. For ``w >= 0`` the result is half the+ axis-angle vector; for ``w < 0`` it is not, because this map tells ``q`` from ``-q``. For the axis-angle vector,+ use :func:`quaternion_to_axis_angle`.Convention:- - the input is the quaternion of shape ``(*, 4)`` in- ``(w, x, y, z)`` order and the output is a 3-vector of shape- ``(*, 3)``, the argument- :func:`~kornia.geometry.conversions.quaternion_log_to_exp` takes- for a **unit** ``q`` on the ``w >= 0`` half of the double cover theresult is ``quaternion_to_axis_angle(q) / 2``. On the ``w < 0`` half the twopart company: this function keeps the sign of ``w``, with the angle- ``atan2(||v||, w)`` in ``[0, pi]``, while+ ``atan2(||u||, w)`` in ``[0, pi]`` for the vector part ``u``, while:func:`~kornia.geometry.conversions.quaternion_to_axis_angle`collapses the double cover. For the ``q`` whose log is ``v``,``quaternion_exp_to_log(-q)`` is ``-(pi - ||v||) * v / ||v||``- ``quaternion_exp_to_log(quaternion_log_to_exp(v))`` returns ``v`` for- ``0 < ||v|| < pi`` and the wrapped ``||v|| - 2 * pi`` above ``pi``- (``pi + 0.5`` comes back as ``-2.6415926535897936``). At exactly+ ``0 < ||v|| < pi``. A longer ``v`` comes back shifted along its own+ direction by a multiple of ``2 * pi`` to a length of at most ``pi``:+ ``pi + 0.5`` and ``3 * pi + 0.5`` both come back as+ ``-2.6415926535897936``, ``2 * pi + 0.5`` as ``0.5``. At exactly``||v|| = pi`` in ``float64`` it collapses to nearly ``0``: ``cos(pi)``rounds to ``-1`` and the vector part falls under the ``eps`` clamp.Both are properties of the map, not defects- ``float16``/``bfloat16`` are computed in ``float32``, so the identityreturns ``[0., 0., 0.]`` at every dtype, and the gradient is finiteat ``w = +-1``, the identity included- - the map is scale-invariant: a non-unit quaternion produces the same- log as its normalisation+ - the map is invariant to a positive scale while ``||u||`` stays at least ``eps``: a non-unit quaternion+ produces the same log as its normalisation, and ``-q`` another oneArgs:quaternion: a tensor containing a quaternion to be converted.@@ -1108,6 +1306,10 @@ def quaternion_exp_to_log(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torcReturn:the quaternion log map of shape ``(*, 3)``.+ See Also:+ - :func:`quaternion_log_to_exp`: the reverse, the exponential map.+ - :func:`quaternion_to_axis_angle`: the axis-angle vector, twice this one for ``w >= 0``.+Example:>>> quaternion = torch.tensor((1., 0., 0., 0.))>>> quaternion_exp_to_log(quaternion, eps=torch.finfo(quaternion.dtype).eps)@@ -1148,20 +1350,24 @@ def quaternion_exp_to_log(quaternion: torch.Tensor, eps: float = 1.0e-8) -> torcdef axis_angle_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor:- r"""Convert an axis angle to a quaternion.+ r"""Convert axis-angle vectors ``(*, 3)`` in radians to ``(w, x, y, z)`` quaternions ``(*, 4)``.- The quaternion vector has components in (w, x, y, z) format.+ ``q = axis_angle_to_quaternion(axis_angle)`` takes rotation vectors ``(*, 3)``, each the unit axis scaled by the+ angle ``theta`` in radians, and returns the unit quaternions ``(cos(theta / 2), sin(theta / 2) * axis)`` of shape+ ``(*, 4)``, real part first. The sign is not canonicalised. For the reverse, use :func:`quaternion_to_axis_angle`;+ for the rotation matrix, :func:`axis_angle_to_rotation_matrix`; from the half-angle log vector,+ :func:`quaternion_log_to_exp`.Adapted from ceres C++ library: ceres-solver/include/ceres/rotation.hConvention:- - the input is the rotation axis scaled by the angle in **radians**,- shape ``(*, 3)``; the output is ``(w, x, y, z)`` with- ``w = cos(theta / 2)``, shape ``(*, 4)`` (see+ - the ``(w, x, y, z)`` layout is spelled out on:func:`~kornia.geometry.conversions.quaternion_to_rotation_matrix`- for the layout)- - nothing is canonicalised, so ``theta > pi`` returns the ``w < 0`` half- of the double cover: ``[2 * pi, 0., 0.]`` gives ``[-1., 0., 0., 0.]``+ - nothing is canonicalised, so ``pi < theta < 3 * pi`` returns the+ ``w < 0`` half of the double cover: ``[2 * pi, 0., 0.]`` gives+ ``[-1., 0., 0., 0.]``. The sign of ``w = cos(theta / 2)`` keeps+ alternating beyond that interval: ``[4 * pi, 0., 0.]`` gives+ ``[1., 0., 0., 0.]``- the round trip with:func:`~kornia.geometry.conversions.quaternion_to_axis_angle` holdsto roundoff for ``0 <= theta <= pi``@@ -1186,6 +1392,14 @@ def axis_angle_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor:- Input: ``(*, 3)`` where `*` means, any number of dimensions- Output: ``(*, 4)``+ See Also:+ - :func:`quaternion_to_axis_angle`: the reverse conversion, quaternion to axis-angle.+ - :func:`axis_angle_to_rotation_matrix`: the rotation matrix of the same vector.+ - :func:`rotation_matrix_to_quaternion`: the quaternion of a rotation matrix instead, its sign not canonical.+ - :func:`quaternion_log_to_exp`: the same quaternion from half the vector, the exponential map.+ - :meth:`~kornia.geometry.quaternion.Quaternion.from_axis_angle`: the same quaternion as a+ :class:`~kornia.geometry.quaternion.Quaternion` object.+Example:>>> axis_angle = torch.tensor((0., 1., 0.))>>> axis_angle_to_quaternion(axis_angle)@@ -1231,7 +1445,11 @@ def axis_angle_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor:@deprecated(replace_with="axis_angle_to_quaternion", version="0.7.0")-def angle_axis_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor: # noqa: D103+def angle_axis_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor:+ """Deprecated: use :func:`~kornia.geometry.conversions.axis_angle_to_quaternion`.++ Each call emits a ``DeprecationWarning`` (deprecated since 0.7.0) and returns what the new name returns.+ """return axis_angle_to_quaternion(axis_angle)@@ -1241,16 +1459,18 @@ def angle_axis_to_quaternion(axis_angle: torch.Tensor) -> torch.Tensor: # noqa:def euler_from_quaternion(w: torch.Tensor, x: torch.Tensor, y: torch.Tensor, z: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:- """Convert a quaternion coefficients to Euler angles.+ r"""Convert a quaternion given as four tensors w, x, y, z to roll, pitch and yaw in radians.++ ``roll, pitch, yaw = euler_from_quaternion(w, x, y, z)`` takes the four quaternion components as **separate**+ tensors, not as one ``(*, 4)`` tensor, and returns three tensors of their shape: the angles in radians about ``x``,+ ``y`` and ``z`` of the rotation ``Rz(yaw) @ Ry(pitch) @ Rx(roll)``, with ``roll`` and ``yaw`` in+ :math:`[-\pi, \pi]` and ``pitch`` in :math:`[-\pi/2, \pi/2]`. For a ``(*, 4)`` quaternion ``q``, pass+ ``*q.unbind(-1)``. For the reverse, use :func:`quaternion_from_euler`.Convention:- - the four quaternion coefficients are passed as **separate** tensors in- ``(w, x, y, z)`` order, not as one ``(*, 4)`` tensor, and their- shapes must be exactly equal: broadcastable shapes such as ``(2,)``- and ``(1,)`` raise ``BaseError``- - the return is a tuple of three tensors ``(roll, pitch, yaw)`` in- **radians**, with ``roll`` about ``x``, ``pitch`` about ``y`` and- ``yaw`` about ``z``; the composition they stand for is documented on+ - the four shapes must be exactly equal: broadcastable shapes such as+ ``(2,)`` and ``(1,)`` raise ``BaseError``+ - the composition the triple stands for is documented on:func:`~kornia.geometry.conversions.quaternion_from_euler`- away from ``|pitch| = pi/2`` it returns the triple of the inputrotation folded into ``roll, yaw`` in ``[-pi, pi]`` ā the interval is@@ -1292,7 +1512,12 @@ def euler_from_quaternion(z: quaternion :math:`q_z` coefficient.Return:- A tuple with euler angles`roll`, `pitch`, `yaw`.+ A tuple with the euler angles ``roll``, ``pitch``, ``yaw`` in radians.++ See Also:+ - :func:`quaternion_from_euler`: the reverse conversion, Euler angles to quaternion components.+ - :meth:`~kornia.geometry.quaternion.Quaternion.to_euler`: the same angles of a+ :class:`~kornia.geometry.quaternion.Quaternion` object."""KORNIA_CHECK(w.shape == x.shape)@@ -1370,7 +1595,12 @@ def euler_from_quaternion(def quaternion_from_euler(roll: torch.Tensor, pitch: torch.Tensor, yaw: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:- """Convert Euler angles to quaternion coefficients.+ """Convert roll, pitch and yaw in radians to a quaternion returned as four tensors w, x, y, z.++ ``w, x, y, z = quaternion_from_euler(roll, pitch, yaw)`` takes the angles in radians about ``x``, ``y`` and ``z``+ as tensors of one shape and returns, as four tensors of that shape, the components of the unit quaternion of the+ rotation ``Rz(yaw) @ Ry(pitch) @ Rx(roll)``, real part first. For one ``(*, 4)`` tensor, stack them with+ ``torch.stack((w, x, y, z), -1)``. For the reverse, use :func:`euler_from_quaternion`.Convention:- ``roll``, ``pitch`` and ``yaw`` are in **radians** and turn about@@ -1392,7 +1622,12 @@ def quaternion_from_euler(yaw: the yaw euler angle.Return:- A tuple with quaternion coefficients in order of `wxyz`.+ A tuple with quaternion coefficients in order of ``wxyz``.++ See Also:+ - :func:`euler_from_quaternion`: the reverse conversion, quaternion components to Euler angles.+ - :meth:`~kornia.geometry.quaternion.Quaternion.from_euler`: the same quaternion as a+ :class:`~kornia.geometry.quaternion.Quaternion` object."""KORNIA_CHECK(roll.shape == pitch.shape)@@ -1424,7 +1659,18 @@ def quaternion_from_euler(def normalize_pixel_coordinates(pixel_coordinates: torch.Tensor, height: int, width: int, eps: float = 1e-8) -> torch.Tensor:- r"""Map pixel coordinates so that the first and last pixel of each axis become -1 and 1.+ r"""Map 2D ``(x, y)`` pixel coordinates to ``[-1, 1]``, corner-aligned.++ ``coords_norm = normalize_pixel_coordinates(coords, height, width)`` takes pixel coordinates ``(*, 2)`` in+ ``(x, y)`` order and the image size in ``(height, width)`` order, and returns them as normalized+ :math:`[-1, 1]` coordinates, corner-aligned: the centres of the first and last pixel of each axis map to ``-1``+ and ``1``. This is the ``align_corners=True`` convention, and there is no other choice here: for the half-pixel+ convention of ``align_corners=False``, apply+ ``normal_transform_pixel(height, width, device=..., dtype=..., align_corners=False)``, on the device and in the+ floating dtype of the coordinates (cast integer ones first), with+ :func:`~kornia.geometry.linalg.transform_points`, which needs a point axis, ``(*, N, 2)`` (write a single ``(2,)``+ coordinate as ``coords[None]``). For the reverse, use+ :func:`denormalize_pixel_coordinates`; for ``(*, 3)`` voxel coordinates, :func:`normalize_pixel_coordinates3d`.Convention:See :doc:`Conventions & Pitfalls </get-started/conventions>` for the library-wide pixel-centre,@@ -1434,17 +1680,16 @@ def normalize_pixel_coordinates(indexes columns and is scaled by ``width``, ``y`` indexes rows and isscaled by ``height``. The positional argument order is the other wayround ā ``(pixel_coordinates, height, width)``- - the mapping is **corner-aligned**,- ``x_norm = 2 * x / (width - 1) - 1``: for ``width = 4`` the columns- ``0``, ``1``, ``3`` map to ``-1``, ``-1/3``, ``+1``. This is the- ``align_corners=True`` convention;- :func:`torch.nn.functional.grid_sample` at its default- ``align_corners=False`` would instead place the first and last of- those values at pixels ``-0.5`` and ``3.5`` ā half a pixel outside- the image on each side ā so ``align_corners=True`` must be passed- explicitly when feeding this output to it. Note that kornia's own- :func:`~kornia.geometry.transform.remap` resolves- ``align_corners=None`` to ``False``+ - the mapping is ``x_norm = 2 * x / (width - 1) - 1``: for+ ``width = 4`` the columns ``0``, ``1``, ``3`` map to ``-1``,+ ``-1/3``, ``+1``. :func:`torch.nn.functional.grid_sample` at its+ default ``align_corners=False`` would instead place the first and+ last of those values at pixels ``-0.5`` and ``3.5`` ā half a pixel+ outside the image on each side ā so ``align_corners=True`` must be+ passed explicitly when feeding this output to it. kornia's own+ :func:`~kornia.geometry.transform.remap` normalizes pixel maps with+ this function but resolves ``align_corners=None`` to ``False``+ (`#4504 <https://github.com/kornia/kornia/issues/4504>`_)- a singleton axis uses scale ``1`` and offset ``0``: its only validpixel coordinate maps to the normalized centre, and the unit-scaleextension keeps this function exactly invertible outside that lone@@ -1464,13 +1709,20 @@ def normalize_pixel_coordinates(Args:pixel_coordinates: the grid with pixel coordinates. Shape can be ``(*, 2)``.- height: the maximum height in the y-axis.- width: the maximum width in the x-axis.+ height: the image height, the number of rows, which scales ``y``.+ width: the image width, the number of columns, which scales ``x``.eps: deprecated compatibility parameter. It is ignored.Return:the normalized pixel coordinates with shape ``(*, 2)``.+ See Also:+ - :func:`denormalize_pixel_coordinates`: the reverse map, :math:`[-1, 1]` back to pixels.+ - :func:`normalize_pixel_coordinates3d`: the 3D form, for ``(*, 3)`` coordinates in ``(d, x, y)`` order.+ - :func:`normal_transform_pixel`: the same map as a ``(1, 3, 3)`` matrix, with an ``align_corners`` choice.+ - :func:`~kornia.geometry.grid.create_meshgrid`: the whole ``(1, H, W, 2)`` pixel grid, normalized the same+ way by default.+Examples:>>> coords = torch.tensor([[50., 100.]])>>> normalize_pixel_coordinates(coords, 100, 50)@@ -1523,32 +1775,35 @@ def normalize_pixel_coordinates(def denormalize_pixel_coordinates(pixel_coordinates: torch.Tensor, height: int, width: int, eps: float = 1e-8) -> torch.Tensor:- r"""Denormalize pixel coordinates.+ r"""Map 2D ``(x, y)`` coordinates from ``[-1, 1]`` back to pixels, corner-aligned.- The input is assumed to be -1 if on extreme left, 1 if on extreme right (x = w-1).+ ``coords = denormalize_pixel_coordinates(coords_norm, height, width)`` is the inverse of+ :func:`normalize_pixel_coordinates`: it takes corner-aligned coordinates ``(*, 2)`` in ``(x, y)`` order, with+ ``-1`` and ``1`` at the centres of the first and last pixel of each axis, and the image size in+ ``(height, width)`` order, and returns the pixel coordinates ``(*, 2)``, ``x = (width - 1) * (x_norm + 1) / 2``,+ so that ``1`` maps to ``x = width - 1``. For ``(*, 3)`` voxel coordinates, use+ :func:`denormalize_pixel_coordinates3d`.Convention:See :doc:`Conventions & Pitfalls </get-started/conventions>` for the library-wide pixel-centre,normalized-coordinate and ``align_corners`` conventions.- - the inverse of- :func:`~kornia.geometry.conversions.normalize_pixel_coordinates`,- ``x = (width - 1) * (x_norm + 1) / 2``, with the same ``(x, y)``- component order and the same ``(pixel_coordinates, height, width)``- positional order-- singleton axes, non-positive-size validation, tracing assumptions, andthe deprecated ``eps`` parameter follow the normalization functionArgs:pixel_coordinates: the normalized grid coordinates. Shape can be ``(*, 2)``.- height: the maximum height in the y-axis.- width: the maximum width in the x-axis.+ height: the image height, the number of rows, which scales ``y``.+ width: the image width, the number of columns, which scales ``x``.eps: deprecated compatibility parameter. It is ignored.Return:the denormalized pixel coordinates with shape ``(*, 2)``.+ See Also:+ - :func:`normalize_pixel_coordinates`: the forward map, pixels to :math:`[-1, 1]`.+ - :func:`denormalize_pixel_coordinates3d`: the 3D form, for ``(*, 3)`` coordinates in ``(d, x, y)`` order.+Examples:>>> coords = torch.tensor([[-1., -1.]])>>> denormalize_pixel_coordinates(coords, 100, 50)@@ -1602,31 +1857,41 @@ def denormalize_pixel_coordinates(def normalize_pixel_coordinates3d(pixel_coordinates: torch.Tensor, depth: int, height: int, width: int, eps: float = 1e-8) -> torch.Tensor:- r"""Map 3d pixel coordinates so that the first and last sample of each axis become -1 and 1.+ r"""Map 3D ``(d, x, y)`` voxel coordinates to ``[-1, 1]``, corner-aligned.++ ``coords_norm = normalize_pixel_coordinates3d(coords, depth, height, width)`` takes voxel coordinates ``(*, 3)`` in+ ``(d, x, y)`` order, the order of :func:`~kornia.geometry.grid.create_meshgrid3d`, and returns them as+ normalized :math:`[-1, 1]` coordinates, corner-aligned and unclamped in each axis as+ :func:`normalize_pixel_coordinates` is.+ :func:`normal_transform_pixel3d` reads ``(x, y, z)`` instead, so a grid built for one permutes the axes when fed+ to the other. For the reverse, use :func:`denormalize_pixel_coordinates3d`.Convention:- ``pixel_coordinates`` is ``(*, 3)`` in ``(d, x, y)`` order ā- **depth first, then x, then y**, *not* ``(x, y, z)``. This is the- order :func:`~kornia.geometry.grid.create_meshgrid3d` produces. The+ **depth first, then x, then y**, *not* ``(x, y, z)``. Thethree components are scaled by ``depth - 1``, ``width - 1`` and``height - 1`` respectively, so with ``depth=3, height=5, width=9``the coordinate ``[2., 8., 4.]`` maps to ``[1., 1., 1.]``- the positional argument order is``(pixel_coordinates, depth, height, width)``- - corner-aligned and unclamped in each axis exactly as in- :func:`~kornia.geometry.conversions.normalize_pixel_coordinates`- singleton axes, non-positive-size validation, tracing assumptions, and- the deprecated ``eps`` parameter likewise follow that function+ the deprecated ``eps`` parameter follow+ :func:`~kornia.geometry.conversions.normalize_pixel_coordinates`Args:pixel_coordinates: the grid with pixel coordinates. Shape can be ``(*, 3)``.- depth: the maximum depth in the z-axis.- height: the maximum height in the y-axis.- width: the maximum width in the x-axis.+ depth: the volume depth, which scales ``d``.+ height: the volume height, which scales ``y``.+ width: the volume width, which scales ``x``.eps: deprecated compatibility parameter. It is ignored.Return:- the normalized pixel coordinates.+ the normalized pixel coordinates, ``(*, 3)`` in ``(d, x, y)`` order.++ See Also:+ - :func:`denormalize_pixel_coordinates3d`: the reverse map, :math:`[-1, 1]` back to voxels.+ - :func:`normalize_pixel_coordinates`: the 2D form, for ``(*, 2)`` coordinates in ``(x, y)`` order.+ - :func:`normal_transform_pixel3d`: the same scaling as a ``(1, 4, 4)`` matrix on ``(x, y, z, 1)``."""if pixel_coordinates.shape[-1] != 3:@@ -1689,28 +1954,34 @@ def normalize_pixel_coordinates3d(def denormalize_pixel_coordinates3d(pixel_coordinates: torch.Tensor, depth: int, height: int, width: int, eps: float = 1e-8) -> torch.Tensor:- r"""Denormalize 3d pixel coordinates.+ r"""Map 3D ``(d, x, y)`` coordinates from ``[-1, 1]`` back to voxels, corner-aligned.- Convention:- - the inverse of- :func:`~kornia.geometry.conversions.normalize_pixel_coordinates3d`,- with the same ``(d, x, y)`` component order and the same- ``(pixel_coordinates, depth, height, width)`` positional order: with- ``depth=3, height=5, width=9`` the input ``[0., 0., 0.]`` maps back to- ``[1., 4., 2.]``+ ``coords = denormalize_pixel_coordinates3d(coords_norm, depth, height, width)`` is the inverse of+ :func:`normalize_pixel_coordinates3d`: it takes corner-aligned coordinates ``(*, 3)`` in ``(d, x, y)`` order and+ returns the voxel coordinates ``(*, 3)`` in the same order. For ``(*, 2)`` pixel coordinates, use+ :func:`denormalize_pixel_coordinates`.+ Convention:+ - the ``(d, x, y)`` component order and the+ ``(pixel_coordinates, depth, height, width)`` positional order are+ those of the normalization: with ``depth=3, height=5, width=9`` the+ input ``[0., 0., 0.]`` maps back to ``[1., 4., 2.]``- singleton axes, non-positive-size validation, tracing assumptions, andthe deprecated ``eps`` parameter follow the 2-D counterpartArgs:pixel_coordinates: the normalized grid coordinates. Shape can be ``(*, 3)``.- depth: the maximum depth in the z-axis.- height: the maximum height in the y-axis.- width: the maximum width in the x-axis.+ depth: the volume depth, which scales ``d``.+ height: the volume height, which scales ``y``.+ width: the volume width, which scales ``x``.eps: deprecated compatibility parameter. It is ignored.Return:- the denormalized pixel coordinates.+ the denormalized pixel coordinates, ``(*, 3)`` in ``(d, x, y)`` order.++ See Also:+ - :func:`normalize_pixel_coordinates3d`: the forward map, voxels to :math:`[-1, 1]`.+ - :func:`denormalize_pixel_coordinates`: the 2D form, for ``(*, 2)`` coordinates in ``(x, y)`` order."""if pixel_coordinates.shape[-1] != 3:@@ -1768,12 +2039,16 @@ def denormalize_pixel_coordinates3d(def angle_to_rotation_matrix(angle: torch.Tensor) -> torch.Tensor:- r"""Create a rotation matrix out of angles in degrees.+ r"""Build 2D rotation matrices ``(*, 2, 2)`` from angles in degrees, counter-clockwise as displayed.++ ``R = angle_to_rotation_matrix(angle)`` takes angles of any shape ``(*)`` in **degrees** and returns the matrices+ ``[[cos, sin], [-sin, cos]]`` of shape ``(*, 2, 2)``. Applied to ``(x, y)`` column vectors in image coordinates,+ with ``y`` down, a positive angle turns counter-clockwise as displayed, which is clockwise in a ``y``-up plot. For+ a ``(B, 2, 3)`` matrix that also scales and turns about a centre, use+ :func:`~kornia.geometry.transform.get_rotation_matrix2d`; for an angle in radians, pass ``rad2deg(angle)``.Convention:- - ``angle`` is in **degrees**, shape ``(*)`` in and- ``(*, 2, 2)`` out- - the matrix is ``[[cos, sin], [-sin, cos]]`` with ``det = +1``; for+ - the matrix has ``det = +1``; for``angle = 30`` it is ``[[0.8660, 0.5000], [-0.5000, 0.8660]]``- applied on the left to a column vector ``(x, y)``, ``angle = 30`` maps``(1., 0.)`` to ``(0.8660, -0.5000)`` ā counter-clockwise **as@@ -1788,6 +2063,12 @@ def angle_to_rotation_matrix(angle: torch.Tensor) -> torch.Tensor:Returns:tensor of rotation matrices with shape ``(*, 2, 2)``.+ See Also:+ - :func:`~kornia.geometry.transform.get_rotation_matrix2d`: the ``(B, 2, 3)`` matrix that also scales and+ turns about a centre, in the same sense.+ - :func:`axis_angle_to_rotation_matrix`: a 3D ``(*, 3, 3)`` rotation from an axis-angle vector in radians.+ - :func:`cart2pol`: the polar angle of a point, which this rotation decreases.+Example:>>> input = torch.rand(1, 3) # Nx3>>> output = angle_to_rotation_matrix(input) # Nx3x2x2@@ -1805,15 +2086,22 @@ def normalize_homography(dsize_dst: tuple[int, int],align_corners: bool = True,) -> torch.Tensor:- r"""Express a 2D sourceādestination pixel homography (B, 3, 3) in [-1, 1] coordinates, corner-aligned by default.+ r"""Express a sourceādestination pixel homography ``(B, 3, 3)`` in ``[-1, 1]`` coordinates.++ ``H_norm = normalize_homography(H, dsize_src, dsize_dst)`` takes homographies ``(B, 3, 3)``, or one ``(3, 3)``,+ that map **source pixels to destination pixels**, and the ``(height, width)`` of both images, and returns+ ``N_dst @ H @ inv(N_src)`` ``(B, 3, 3)``: the same maps between normalized :math:`[-1, 1]` coordinates, with ``N``+ from :func:`normal_transform_pixel`. Pass the ``align_corners`` of the sampling that will use the result: for a+ ``(B, 3, 3)`` ``H`` and ``H_norm = normalize_homography(H, img.shape[-2:], dsize, align_corners=a)``,+ ``homography_warp(img, torch.linalg.inv(H_norm), dsize, align_corners=a)`` samples what+ ``warp_perspective(img, H, dsize, align_corners=a)`` samples, since+ :func:`~kornia.geometry.transform.warp_perspective` normalizes a pixel homography itself. For the reverse, use+ :func:`denormalize_homography`; for ``(B, 4, 4)`` voxel homographies, :func:`normalize_homography3d`.Convention:- - the input maps **source pixels to destination pixels** and the output- maps **normalized source to normalized destination**: exactly- ``N_dst @ H @ inv(N_src)`` with ``N`` from- :func:`~kornia.geometry.conversions.normal_transform_pixel`, so- ``dsize_src`` drives the right factor and ``dsize_dst`` the left one.- A ``+2`` pixel shift in ``x`` on a 5-wide image comes back as ``+1.0``.+ - ``dsize_src`` drives the right factor and ``dsize_dst`` the left one:+ a ``+2`` pixel shift in ``x`` on a 5-wide image comes back as ``+1.0``+ (``+0.8`` with ``align_corners=False``).:func:`~kornia.geometry.conversions.denormalize_homography` computesthe reverse ``inv(N_dst) @ H @ N_src``- both ``dsize`` arguments are ``(height, width)``, while the matrix acts@@ -1867,6 +2155,15 @@ def normalize_homography(Returns:the normalized homography of shape ``(B, 3, 3)``.+ See Also:+ - :func:`denormalize_homography`: the reverse, a normalized homography back to pixels.+ - :func:`normalize_homography3d`: the 3D form for ``(B, 4, 4)`` voxel homographies, always corner-aligned.+ - :func:`normal_transform_pixel`: the ``(1, 3, 3)`` normalization matrix ``N`` composed here.+ - :func:`~kornia.geometry.transform.warp_perspective`: warps by the pixel homography, normalizing it with+ this function.+ - :func:`~kornia.geometry.transform.homography_warp`: warps by the inverse of the result, a+ destinationāsource normalized homography.+"""if not isinstance(dst_pix_trans_src_pix, torch.Tensor):raise TypeError(f"Input type is not a torch.Tensor. Got {type(dst_pix_trans_src_pix)}")@@ -1916,14 +2213,19 @@ def normal_transform_pixel(dtype: Optional[torch.dtype] = None,align_corners: bool = True,) -> torch.Tensor:- r"""Compute the (1, 3, 3) matrix that maps 2D pixel coordinates (x, y) to [-1, 1], corner-aligned by default.+ r"""Compute the ``(1, 3, 3)`` matrix that maps 2D pixel coordinates ``(x, y)`` to ``[-1, 1]``.++ ``N = normal_transform_pixel(height, width)`` returns a **single** matrix ``(1, 3, 3)``, never batched, that maps+ the homogeneous pixel coordinates ``(x, y, 1)`` of a ``height`` by ``width`` image to :math:`[-1, 1]`: the+ corner-aligned map of :func:`normalize_pixel_coordinates` by default, the half-pixel map with+ ``align_corners=False``. It is the ``N`` that :func:`normalize_homography` and :func:`denormalize_homography`+ compose with a homography. To normalize points directly, use :func:`normalize_pixel_coordinates`; for+ ``(1, 4, 4)`` volume matrices, :func:`normal_transform_pixel3d`.Convention:- - the result is a **single** matrix of shape ``(1, 3, 3)`` ā never- batched ā acting on homogeneous ``(x, y, 1)`` column vectors, with- ``x`` indexing columns and scaled by ``width`` and ``y`` indexing rows- and scaled by ``height``. The positional argument order is the other- way round, ``(height, width)``+ - ``x`` indexes columns and is scaled by ``width``, ``y`` indexes rows+ and is scaled by ``height``. The positional argument order is the+ other way round, ``(height, width)``- the mapping is **corner-aligned by default**: scale ``2 / (size - 1)``,offset ``-1``, so the pixel *centres* ``0`` and ``size - 1`` map toexactly ``-1`` and ``+1``. ``align_corners=False`` selects the@@ -1981,6 +2283,11 @@ def normal_transform_pixel(Returns:normalized transform with shape ``(1, 3, 3)``.+ See Also:+ - :func:`normal_transform_pixel3d`: the 3D ``(1, 4, 4)`` matrix, always corner-aligned.+ - :func:`normalize_pixel_coordinates`: the same corner-aligned map applied to ``(*, 2)`` points.+ - :func:`normalize_homography`: conjugates a pixel homography with this matrix.+Example:>>> normal_transform_pixel(4, 5)tensor([[[ 0.5000, 0.0000, -1.0000],@@ -2105,16 +2412,21 @@ def normal_transform_pixel3d(device: Optional[torch.device] = None,dtype: Optional[torch.dtype] = None,) -> torch.Tensor:- r"""Compute the (1, 4, 4) matrix that maps 3D voxel coordinates (x, y, z) to [-1, 1], always corner-aligned.+ r"""Compute the ``(1, 4, 4)`` matrix that maps 3D voxel coordinates ``(x, y, z)`` to ``[-1, 1]``.++ ``N = normal_transform_pixel3d(depth, height, width)`` returns a single matrix ``(1, 4, 4)`` that maps the+ homogeneous voxel coordinates ``(x, y, z, 1)`` of a ``depth`` by ``height`` by ``width`` volume to+ :math:`[-1, 1]`, corner-aligned, as :func:`normalize_homography3d` composes it. It has no ``align_corners``+ parameter, so a homography normalized with it suits only ``align_corners=True`` sampling, a restriction the 3D+ warps that use it inherit (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). Its ``(x, y, z)`` order+ differs from the ``(d, x, y)`` of :func:`normalize_pixel_coordinates3d`.Convention:- the 3-D counterpart of:func:`~kornia.geometry.conversions.normal_transform_pixel`: samecorner-aligned ``2 / (size - 1)`` scaling with offset ``-1``, same``dtype=None`` / ``torch.get_default_dtype()`` rule, and likewise- **never batched**. Unlike the 2-D function it has no ``align_corners``- parameter, so it applies that scaling unconditionally.- Singleton axes use the same invertible centre mapping, and zero or+ **never batched**. Singleton axes use the same invertible centre mapping, and zero ornegative sizes raise ``ValueError``. Only the lines below differ- the result has shape ``(1, 4, 4)`` and acts on homogeneous``(x, y, z, 1)`` column vectors with ``x`` scaled by ``width``, ``y``@@ -2151,6 +2463,13 @@ def normal_transform_pixel3d(Returns:normalized transform with shape ``(1, 4, 4)``.+ See Also:+ - :func:`normal_transform_pixel`: the 2D ``(1, 3, 3)`` matrix, which also has the ``align_corners=False``+ map.+ - :func:`normalize_homography3d`: conjugates a voxel homography with this matrix.+ - :func:`normalize_pixel_coordinates3d`: the same scaling applied to ``(*, 3)`` points in ``(d, x, y)``+ order.+Example:>>> normal_transform_pixel3d(2, 4, 5)tensor([[[ 0.5000, 0.0000, 0.0000, -1.0000],@@ -2258,15 +2577,18 @@ def denormalize_homography(dsize_dst: tuple[int, int],align_corners: bool = True,) -> torch.Tensor:- r"""De-normalize a given homography in pixels from [-1, 1] to actual height and width.+ r"""Express a ``[-1, 1]`` homography ``(B, 3, 3)`` as a sourceādestination pixel homography.++ ``H = denormalize_homography(H_norm, dsize_src, dsize_dst)`` is the inverse of :func:`normalize_homography`: it+ takes homographies ``(B, 3, 3)``, or one ``(3, 3)``, from normalized :math:`[-1, 1]` source coordinates to+ normalized destination coordinates, and the ``(height, width)`` of both images, and returns the sourceādestination+ pixel homographies ``inv(N_dst) @ H_norm @ N_src`` ``(B, 3, 3)``. Pass the ``align_corners`` that produced+ ``H_norm``. For ``(B, 4, 4)`` voxel homographies, use :func:`denormalize_homography3d`.Convention:- the mirror of- :func:`~kornia.geometry.conversions.normalize_homography`: the- composition is ``inv(N_dst) @ H @ N_src``, so the input maps- normalized source to normalized destination and the output maps- source pixels to destination pixels. The ``dsize`` roles, the- ``(x, y, 1)`` column vectors, per-sample batching, the+ :func:`~kornia.geometry.conversions.normalize_homography`. The ``dsize``+ roles, the ``(x, y, 1)`` column vectors, per-sample batching, the``align_corners``-selected frames and the shape guard are asdocumented there, including its integer-input warning- ``inv(N_dst)`` goes through ``torch.linalg.inv`` in eager mode (so@@ -2288,6 +2610,11 @@ def denormalize_homography(Returns:the denormalized homography of shape ``(B, 3, 3)``.+ See Also:+ - :func:`normalize_homography`: the forward conversion, pixels to normalized coordinates.+ - :func:`denormalize_homography3d`: the 3D form for ``(B, 4, 4)`` homographies, always corner-aligned.+ - :func:`normal_transform_pixel`: the ``(1, 3, 3)`` normalization matrix ``N`` composed here.+"""if not isinstance(dst_pix_trans_src_pix, torch.Tensor):raise TypeError(f"Input type is not a torch.Tensor. Got {type(dst_pix_trans_src_pix)}")@@ -2334,7 +2661,14 @@ def denormalize_homography(def normalize_homography3d(dst_pix_trans_src_pix: torch.Tensor, dsize_src: tuple[int, int, int], dsize_dst: tuple[int, int, int]) -> torch.Tensor:- r"""Express a 3D sourceādestination voxel homography (B, 4, 4) in [-1, 1] coordinates, always corner-aligned.+ r"""Express a sourceādestination voxel homography ``(B, 4, 4)`` in ``[-1, 1]`` coordinates.++ ``H_norm = normalize_homography3d(H, dsize_src, dsize_dst)`` takes homographies ``(B, 4, 4)``, or one ``(4, 4)``,+ that map **source voxels to destination voxels**, and the ``(depth, height, width)`` of both volumes, and returns+ ``N_dst @ H @ inv(N_src)`` ``(B, 4, 4)`` with ``N`` from :func:`normal_transform_pixel3d`. It has no+ ``align_corners`` parameter: the result suits only ``align_corners=True`` sampling+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For the reverse, use :func:`denormalize_homography3d`;+ for ``(B, 3, 3)`` pixel homographies, :func:`normalize_homography`.Convention:- the 3-D counterpart of@@ -2343,8 +2677,7 @@ def normalize_homography3d(direction re-expressed in normalized frames, same ``dsize_src`` on theright and ``dsize_dst`` on the left, and corner-aligned frames ā with:func:`~kornia.geometry.conversions.normal_transform_pixel3d` in place- of the 2-D helper. Unlike the 2-D function it has no ``align_corners``- parameter, so its frames are corner-aligned unconditionally. It is+ of the 2-D helper. It is**not** the 2-D function with wider matrices, though: the shapes andā least visibly ā the inversion routine differ, as the bullets belowrecord@@ -2374,6 +2707,11 @@ def normalize_homography3d(Shape:Output: ``(B, 4, 4)``+ See Also:+ - :func:`denormalize_homography3d`: the reverse, a normalized voxel homography back to voxels.+ - :func:`normalize_homography`: the 2D form for ``(B, 3, 3)`` pixel homographies, with ``align_corners``.+ - :func:`normal_transform_pixel3d`: the ``(1, 4, 4)`` normalization matrix ``N`` composed here.+"""if not isinstance(dst_pix_trans_src_pix, torch.Tensor):raise TypeError(f"Input type is not a torch.Tensor. Got {type(dst_pix_trans_src_pix)}")@@ -2410,14 +2748,18 @@ def normalize_homography3d(def denormalize_homography3d(dst_pix_trans_src_pix: torch.Tensor, dsize_src: tuple[int, int, int], dsize_dst: tuple[int, int, int]) -> torch.Tensor:- r"""De-normalize a given 3D homography from [-1, 1] to actual depth, height and width.+ r"""Express a ``[-1, 1]`` homography ``(B, 4, 4)`` as a sourceādestination voxel homography.++ ``H = denormalize_homography3d(H_norm, dsize_src, dsize_dst)`` is the inverse of :func:`normalize_homography3d`: it+ takes real floating homographies ``(B, 4, 4)``, or one ``(4, 4)``, from normalized :math:`[-1, 1]` source+ coordinates to normalized destination coordinates, and the ``(depth, height, width)`` of both volumes, and returns+ the sourceādestination voxel homographies ``inv(N_dst) @ H_norm @ N_src`` ``(B, 4, 4)``, always corner-aligned+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For ``(B, 3, 3)`` pixel homographies, use+ :func:`denormalize_homography`.Convention:- the mirror of- :func:`~kornia.geometry.conversions.normalize_homography3d`: the- composition is ``inv(N_dst) @ H @ N_src``, so the input maps- normalized source to normalized destination and the output maps- source voxels to destination voxels. The ``(depth, height, width)``+ :func:`~kornia.geometry.conversions.normalize_homography3d`. The ``(depth, height, width)````dsize`` roles, the ``(x, y, z, 1)`` column vectors, per-samplebatching, the corner-aligned frames and the shape guard are asdocumented there@@ -2450,6 +2792,11 @@ def denormalize_homography3d(Shape:Output: ``(B, 4, 4)``+ See Also:+ - :func:`normalize_homography3d`: the forward conversion, voxels to normalized coordinates.+ - :func:`denormalize_homography`: the 2D form for ``(B, 3, 3)`` homographies, with ``align_corners``.+ - :func:`normal_transform_pixel3d`: the ``(1, 4, 4)`` normalization matrix ``N`` composed here.+"""if not isinstance(dst_pix_trans_src_pix, torch.Tensor):raise TypeError(f"Input type is not a torch.Tensor. Got {type(dst_pix_trans_src_pix)}")@@ -2491,7 +2838,16 @@ def denormalize_homography3d(def normalize_points_with_intrinsics(point_2d: torch.Tensor, camera_matrix: torch.Tensor) -> torch.Tensor:- """Normalize points with intrinsics. Useful for conversion of keypoints to be used with essential matrix.+ """Convert pixel coordinates (u, v) to normalized camera coordinates (x, y) with the intrinsics K.++ ``xy = normalize_points_with_intrinsics(uv, K)`` takes pixel points ``(*, 2)`` and pinhole intrinsics+ ``(*, 3, 3)`` and returns the normalized camera coordinates ``((u - cx) / fx, (v - cy) / fy)``, ``(*, 2)``: the+ point where the ray of each pixel meets the ``z = 1`` plane, the form an essential matrix relates. A+ ``(B, 3, 3)`` ``K`` pairs camera ``b`` with ``uv[b]`` for ``(B, 2)`` and ``(B, N, 2)`` points; for+ ``(B, H, W, 2)`` points pass ``K[:, None, None]``, since the cameras otherwise line up with the ``H`` axis+ (`#6409 <https://github.com/kornia/kornia/issues/6409>`_). For the reverse, use+ :func:`denormalize_points_with_intrinsics`; for 3D points at a depth,+ :func:`~kornia.geometry.camera.perspective.unproject_points`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the camera intrinsics and@@ -2500,10 +2856,8 @@ def normalize_points_with_intrinsics(point_2d: torch.Tensor, camera_matrix: torc- ``point_2d`` is ``(*, 2)`` in pixel ``(u, v)`` order and``camera_matrix`` is a row-major pinhole ``(*, 3, 3)`` with``fx = K[0, 0]``, ``fy = K[1, 1]``, ``cx = K[0, 2]``, ``cy = K[1, 2]``- - the output is the calibrated camera coordinate- ``x = (u - cx) / fx``, ``y = (v - cy) / fy``: with- ``K = [[100., 0., 320.], [0., 200., 240.], [0., 0., 1.]]`` the pixel- ``(420., 440.)`` maps to ``(1., 1.)``+ - with ``K = [[100., 0., 320.], [0., 200., 240.], [0., 0., 1.]]`` the+ pixel ``(420., 440.)`` maps to ``(1., 1.)``- the skew entry ``K[0, 1]`` is **ignored**; only the two diagonalentries and the ``[:2, 2]`` column are read. This is a design choice:kornia's functional pinhole model is zero-skew, like OpenCV's, and@@ -2516,7 +2870,14 @@ def normalize_points_with_intrinsics(point_2d: torch.Tensor, camera_matrix: torccamera_matrix: tensor containing the intrinsics camera matrix. The tensor shape must be ``(*, 3, 3)``.Returns:- tensor of normalized camera coordinates :math:`(x, y)` with shape ``(*, 2)``.+ tensor of normalized camera coordinates :math:`(x, y)` with shape ``(*, 2)``, the leading dimensions+ broadcast from both inputs.++ See Also:+ - :func:`denormalize_points_with_intrinsics`: the reverse, normalized camera coordinates back to pixels.+ - :func:`~kornia.geometry.camera.perspective.unproject_points`: the 3D camera-frame points of pixels at a+ given depth.+ - :func:`normalize_pixel_coordinates`: a different normalization, to :math:`[-1, 1]` by the image size.Example:>>> _ = torch.manual_seed(0)@@ -2541,17 +2902,22 @@ def normalize_points_with_intrinsics(point_2d: torch.Tensor, camera_matrix: torcdef denormalize_points_with_intrinsics(point_2d_norm: torch.Tensor, camera_matrix: torch.Tensor) -> torch.Tensor:- """Denormalize points with intrinsics. Useful for converting normalized camera points back to pixels.+ """Convert normalized camera coordinates (x, y) to pixel coordinates (u, v) with the intrinsics K.++ ``uv = denormalize_points_with_intrinsics(xy, K)`` is the inverse of :func:`normalize_points_with_intrinsics`: it+ takes normalized camera coordinates ``(*, 2)``, points on the ``z = 1`` plane, and pinhole intrinsics+ ``(*, 3, 3)``, and returns the pixels ``(x * fx + cx, y * fy + cy)``, ``(*, 2)``. ``K`` broadcasts against the+ points as in that function: for ``(B, H, W, 2)`` points pass ``K[:, None, None]``+ (`#6409 <https://github.com/kornia/kornia/issues/6409>`_). To project 3D camera-frame points instead, use+ :func:`~kornia.geometry.camera.perspective.project_points`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the camera intrinsics andpixel-centre conventions used by these coordinates.- - the inverse of+ - the ``K`` layout is documented on:func:`~kornia.geometry.conversions.normalize_points_with_intrinsics`,- which documents the ``K`` layout: ``u = x * fx + cx`` and- ``v = y * fy + cy``, and the skew entry ``K[0, 1]`` is ignored here- too+ and the skew entry ``K[0, 1]`` is ignored here tooArgs:point_2d_norm: tensor containing the 2d points in normalized camera coordinates. The shape of the tensor can@@ -2559,7 +2925,12 @@ def denormalize_points_with_intrinsics(point_2d_norm: torch.Tensor, camera_matricamera_matrix: tensor containing the intrinsics camera matrix. The tensor shape must be ``(*, 3, 3)``.Returns:- tensor of :math:`(u, v)` pixel coordinates with shape ``(*, 2)``.+ tensor of :math:`(u, v)` pixel coordinates with shape ``(*, 2)``, the leading dimensions broadcast from both+ inputs.++ See Also:+ - :func:`normalize_points_with_intrinsics`: the forward conversion, pixels to normalized camera coordinates.+ - :func:`~kornia.geometry.camera.perspective.project_points`: the pixels of 3D camera-frame points.Example:>>> _ = torch.manual_seed(0)@@ -2583,7 +2954,14 @@ def denormalize_points_with_intrinsics(point_2d_norm: torch.Tensor, camera_matridef Rt_to_matrix4x4(R: torch.Tensor, t: torch.Tensor) -> torch.Tensor:- r"""Combine 3x3 rotation matrix R and 3x1 translation vector t into 4x4 extrinsics.+ r"""Pack rotations ``(B, 3, 3)`` and translations ``(B, 3, 1)`` into ``(B, 4, 4)`` pose matrices.++ ``M = Rt_to_matrix4x4(R, t)`` places each rotation ``(B, 3, 3)`` in the top-left block and its translation+ ``(B, 3, 1)`` in the last column of a ``(B, 4, 4)`` matrix with the bottom row ``[0, 0, 0, 1]``, so that ``M``+ maps ``x`` to ``R @ x + t``. It checks neither that ``R`` is a rotation nor which way the pose goes:+ camera-to-world and world-to-camera poses pack the same way. For the reverse, use :func:`matrix4x4_to_Rt`; for the+ ``(*, 3, 4)`` matrix ``[R | t]`` without the bottom row,+ :func:`~kornia.geometry.transform.projection_from_Rt`.Convention:- ``t`` goes in the **last column** and the appended bottom row is@@ -2623,6 +3001,13 @@ def Rt_to_matrix4x4(R: torch.Tensor, t: torch.Tensor) -> torch.Tensor:Returns:the extrinsics ``(B, 4, 4)``.+ See Also:+ - :func:`matrix4x4_to_Rt`: the reverse, splitting a ``(B, 4, 4)`` matrix into ``R`` and ``t``.+ - :func:`camtoworld_to_worldtocam_Rt`: inverts a rigid ``(R, t)`` pose before or after packing.+ - :func:`~kornia.geometry.transform.projection_from_Rt`: the ``(*, 3, 4)`` matrix ``[R | t]``, without the+ bottom row.+ - :func:`convert_affinematrix_to_homography3d`: the same ``(B, 4, 4)`` from a ``(B, 3, 4)`` matrix.+Example:>>> R, t = torch.eye(3)[None], torch.ones(3).reshape(1, 3, 1)>>> Rt_to_matrix4x4(R, t)@@ -2642,14 +3027,16 @@ def Rt_to_matrix4x4(R: torch.Tensor, t: torch.Tensor) -> torch.Tensor:def matrix4x4_to_Rt(extrinsics: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert 4x4 extrinsics into 3x3 rotation matrix R and 3x1 translation vector t.+ r"""Split ``(B, 4, 4)`` pose matrices into rotation ``(B, 3, 3)`` and translation ``(B, 3, 1)`` views.++ ``R, t = matrix4x4_to_Rt(M)`` returns the top-left ``(B, 3, 3)`` block and the first three rows ``(B, 3, 1)`` of+ the last column of each ``(B, 4, 4)`` matrix, as views of ``M``; the bottom row is ignored. It is the inverse of+ :func:`Rt_to_matrix4x4`.Convention:- the inverse of:func:`~kornia.geometry.conversions.Rt_to_matrix4x4`, with the same- layout ā ``R`` is the top-left 3x3 block and ``t`` the first three- rows of the last column ā and the same frame-agnosticism. Only the- lines below differ+ layout and the same frame-agnosticism. Only the lines below differ- the input shape is strictly ``(B, 4, 4)``; an unbatched ``(4, 4)``,a ``(B, 3, 3)`` and extra leading dimensions each raise ``ShapeError``- the **bottom row is ignored**: a projective 4x4 is silently truncated@@ -2668,6 +3055,11 @@ def matrix4x4_to_Rt(extrinsics: torch.Tensor) -> tuple[torch.Tensor, torch.TensoR: Rotation matrix, ``(B, 3, 3)``.t: Translation matrix ``(B, 3, 1)``.+ See Also:+ - :func:`Rt_to_matrix4x4`: the reverse, packing ``R`` and ``t`` with the bottom row ``[0, 0, 0, 1]``.+ - :func:`~kornia.geometry.linalg.inverse_transformation`: the rigid inverse of the ``(B, 4, 4)`` matrix+ itself.+Example:>>> ext = torch.eye(4)[None]>>> matrix4x4_to_Rt(ext)@@ -2703,10 +3095,15 @@ def _integer_safe_matmul(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:def camtoworld_graphics_to_vision_4x4(extrinsics_graphics: torch.Tensor) -> torch.Tensor:- r"""Convert a (B, 4, 4) camera-to-world matrix from the graphics (OpenGL) frame to the vision (OpenCV) frame.+ r"""Convert ``(B, 4, 4)`` camera-to-world poses from graphics (OpenGL) to vision (OpenCV) camera axes.- I.e. flips y and z axis. Graphics convention: [+x, +y, +z] == [right, up, backwards].- Vision convention: [+x, +y, +z] == [right, down, forwards].+ ``M_vision = camtoworld_graphics_to_vision_4x4(M_graphics)`` takes camera-to-world poses ``(B, 4, 4)`` whose+ camera axes follow the graphics convention, ``[+x, +y, +z] == [right, up, backwards]``, and returns them with the+ vision camera axes, ``[right, down, forwards]``: it negates the camera ``y`` and ``z`` axes, the second and third+ columns of the rotation block, and leaves the camera centre unchanged. A world-to-camera pose needs the flip on the+ other side, which this function does not make: invert it to camera-to-world with+ :func:`~kornia.geometry.linalg.inverse_transformation`, convert, and invert the result back. For a pose given as+ ``(R, t)``, use :func:`camtoworld_graphics_to_vision_Rt`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the graphics/vision camera@@ -2755,6 +3152,11 @@ def camtoworld_graphics_to_vision_4x4(extrinsics_graphics: torch.Tensor) -> torcReturns:extrinsics: pose matrix ``(B, 4, 4)``.+ See Also:+ - :func:`camtoworld_vision_to_graphics_4x4`: the reverse direction, the same arithmetic.+ - :func:`camtoworld_graphics_to_vision_Rt`: the same conversion for a pose given as ``(R, t)``.+ - :func:`camtoworld_to_worldtocam_Rt`: inverts a camera-to-world pose to world-to-camera.+Example:>>> ext = torch.eye(4)[None]>>> camtoworld_graphics_to_vision_4x4(ext)@@ -2774,10 +3176,12 @@ def camtoworld_graphics_to_vision_4x4(extrinsics_graphics: torch.Tensor) -> torcdef camtoworld_graphics_to_vision_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert a camera-to-world pose given as (R, t) from the graphics (OpenGL) frame to the vision (OpenCV) frame.+ r"""Convert camera-to-world ``(R, t)`` poses from graphics (OpenGL) to vision (OpenCV) camera axes.- I.e. flips y and z axis. Graphics convention: [+x, +y, +z] == [right, up, backwards].- Vision convention: [+x, +y, +z] == [right, down, forwards].+ ``R_vision, t = camtoworld_graphics_to_vision_Rt(R, t)`` is :func:`camtoworld_graphics_to_vision_4x4` for a+ camera-to-world pose given as rotations ``(B, 3, 3)`` and camera centres ``(B, 3, 1)``: it negates the camera+ ``y`` and ``z`` axes, the second and third columns of ``R``, to turn graphics camera axes+ ``[right, up, backwards]`` into vision ones ``[right, down, forwards]``, and returns ``t`` unchanged.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the graphics/vision camera@@ -2793,8 +3197,6 @@ def camtoworld_graphics_to_vision_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[- the shapes are strictly ``(B, 3, 3)`` and ``(B, 3, 1)`` in andout, with the same-batch-size rule and the integer-input ``ValueError`` of:func:`~kornia.geometry.conversions.Rt_to_matrix4x4`- - ``t`` is returned unchanged; only ``R`` has its second and third- columns negatedArgs:R: Rotation matrix, ``(B, 3, 3)``.@@ -2804,6 +3206,11 @@ def camtoworld_graphics_to_vision_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[R: Rotation matrix, ``(B, 3, 3)``.t: Translation matrix ``(B, 3, 1)``.+ See Also:+ - :func:`camtoworld_vision_to_graphics_Rt`: the reverse direction, the same arithmetic.+ - :func:`camtoworld_graphics_to_vision_4x4`: the same conversion for a ``(B, 4, 4)`` matrix.+ - :func:`camtoworld_to_worldtocam_Rt`: inverts a camera-to-world ``(R, t)`` to world-to-camera.+Example:>>> R, t = torch.eye(3)[None], torch.ones(3).reshape(1, 3, 1)>>> camtoworld_graphics_to_vision_Rt(R, t)@@ -2822,10 +3229,13 @@ def camtoworld_graphics_to_vision_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[def camtoworld_vision_to_graphics_4x4(extrinsics_vision: torch.Tensor) -> torch.Tensor:- r"""Convert a (B, 4, 4) camera-to-world matrix from the vision (OpenCV) frame to the graphics (OpenGL) frame.+ r"""Convert ``(B, 4, 4)`` camera-to-world poses from vision (OpenCV) to graphics (OpenGL) camera axes.- I.e. flips y and z axis. Graphics convention: [+x, +y, +z] == [right, up, backwards].- Vision convention: [+x, +y, +z] == [right, down, forwards].+ ``M_graphics = camtoworld_vision_to_graphics_4x4(M_vision)`` takes camera-to-world poses ``(B, 4, 4)`` whose+ camera axes follow the vision convention, ``[+x, +y, +z] == [right, down, forwards]``, and returns them with the+ graphics camera axes, ``[right, up, backwards]``, by negating the camera ``y`` and ``z`` axes. The flip is its own+ inverse, so this is the same arithmetic as :func:`camtoworld_graphics_to_vision_4x4`; the name records the+ direction. For a pose given as ``(R, t)``, use :func:`camtoworld_vision_to_graphics_Rt`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the graphics/vision camera@@ -2848,6 +3258,10 @@ def camtoworld_vision_to_graphics_4x4(extrinsics_vision: torch.Tensor) -> torch.Returns:extrinsics: pose matrix ``(B, 4, 4)``.+ See Also:+ - :func:`camtoworld_graphics_to_vision_4x4`: the forward direction, which documents the flip.+ - :func:`camtoworld_vision_to_graphics_Rt`: the same conversion for a pose given as ``(R, t)``.+Example:>>> ext = torch.eye(4)[None]>>> camtoworld_vision_to_graphics_4x4(ext)@@ -2867,10 +3281,12 @@ def camtoworld_vision_to_graphics_4x4(extrinsics_vision: torch.Tensor) -> torch.def camtoworld_vision_to_graphics_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert a camera-to-world pose given as (R, t) from the vision (OpenCV) frame to the graphics (OpenGL) frame.+ r"""Convert camera-to-world ``(R, t)`` poses from vision (OpenCV) to graphics (OpenGL) camera axes.- I.e. flips y and z axis. Graphics convention: [+x, +y, +z] == [right, up, backwards].- Vision convention: [+x, +y, +z] == [right, down, forwards].+ ``R_graphics, t = camtoworld_vision_to_graphics_Rt(R, t)`` is :func:`camtoworld_vision_to_graphics_4x4` for a+ camera-to-world pose given as rotations ``(B, 3, 3)`` and camera centres ``(B, 3, 1)``: it negates the second and+ third columns of ``R`` and returns ``t`` unchanged, the same arithmetic as+ :func:`camtoworld_graphics_to_vision_Rt`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the graphics/vision camera@@ -2894,6 +3310,10 @@ def camtoworld_vision_to_graphics_Rt(R: torch.Tensor, t: torch.Tensor) -> tuple[R: Rotation matrix, ``(B, 3, 3)``.t: Translation matrix ``(B, 3, 1)``.+ See Also:+ - :func:`camtoworld_graphics_to_vision_Rt`: the forward direction, the same arithmetic.+ - :func:`camtoworld_vision_to_graphics_4x4`: the same conversion for a ``(B, 4, 4)`` matrix.+Example:>>> R, t = torch.eye(3)[None], torch.ones(3).reshape(1, 3, 1)>>> camtoworld_vision_to_graphics_Rt(R, t)@@ -2959,10 +3379,14 @@ def _check_is_rotation(R: torch.Tensor, fn_name: str) -> None:def camtoworld_to_worldtocam_Rt(R: torch.Tensor, t: torch.Tensor, check_rotation: bool = False) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert camtoworld to worldtocam frame used in Colmap.+ r"""Invert camera-to-world poses (R, t) into the world-to-camera poses that Colmap stores.- See- long-url: https://colmap.github.io/format.html#output-format+ ``R_w2c, t_w2c = camtoworld_to_worldtocam_Rt(R, t)`` takes camera-to-world rotations ``(B, 3, 3)`` and camera+ centres ``(B, 3, 1)`` and returns the rigid inverse ``(R^T, -R^T @ t)``: the world-to-camera pose of+ `Colmap's images.txt <https://colmap.github.io/format.html#output-format>`_. It transposes rather than inverts, so+ ``R`` must be a rotation; pass ``check_rotation=True`` to verify it. The same formula undoes it, under the name+ :func:`worldtocam_to_camtoworld_Rt`; for a ``(B, 4, 4)`` matrix, use+ :func:`~kornia.geometry.linalg.inverse_transformation`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for camera-to-world versus@@ -3007,6 +3431,12 @@ def camtoworld_to_worldtocam_Rt(Rinv: Rotation matrix, ``(B, 3, 3)``.tinv: Translation matrix ``(B, 3, 1)``.+ See Also:+ - :func:`worldtocam_to_camtoworld_Rt`: the reverse direction, the same formula.+ - :func:`~kornia.geometry.linalg.inverse_transformation`: the same rigid inverse of a ``(B, 4, 4)`` matrix.+ - :func:`camtoworld_graphics_to_vision_Rt`: changes the camera axes of a camera-to-world pose instead.+ - :func:`ARKitQTVecs_to_ColmapQTVecs`: an ARKit quaternion pose to Colmap's, through this inversion.+Example:>>> R, t = torch.eye(3)[None], torch.ones(3).reshape(1, 3, 1)>>> camtoworld_to_worldtocam_Rt(R, t)@@ -3033,7 +3463,13 @@ def camtoworld_to_worldtocam_Rt(def worldtocam_to_camtoworld_Rt(R: torch.Tensor, t: torch.Tensor, check_rotation: bool = False) -> tuple[torch.Tensor, torch.Tensor]:- r"""Convert worldtocam frame used in Colmap to camtoworld.+ r"""Invert world-to-camera poses (R, t), as Colmap stores them, into camera-to-world poses.++ ``R_c2w, t_c2w = worldtocam_to_camtoworld_Rt(R, t)`` takes world-to-camera rotations ``(B, 3, 3)`` and+ translations ``(B, 3, 1)``, as in Colmap's ``images.txt``, and returns the rigid inverse ``(R^T, -R^T @ t)``: the+ camera-to-world pose, whose translation is the camera centre in world coordinates. It is the same formula as+ :func:`camtoworld_to_worldtocam_Rt`, which documents it; ``R`` must be a rotation, verified with+ ``check_rotation=True``.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for camera-to-world versus@@ -3061,6 +3497,11 @@ def worldtocam_to_camtoworld_Rt(Rinv: Rotation matrix, ``(B, 3, 3)``.tinv: Translation matrix ``(B, 3, 1)``.+ See Also:+ - :func:`camtoworld_to_worldtocam_Rt`: the reverse direction, the same formula.+ - :func:`~kornia.geometry.linalg.inverse_transformation`: the same rigid inverse of a ``(B, 4, 4)`` matrix.+ - :func:`ColmapQTVecs_to_ARKitQTVecs`: a Colmap quaternion pose to ARKit's, through this inversion.+Example:>>> R, t = torch.eye(3)[None], torch.ones(3).reshape(1, 3, 1)>>> worldtocam_to_camtoworld_Rt(R, t)@@ -3087,7 +3528,12 @@ def worldtocam_to_camtoworld_Rt(def ARKitQTVecs_to_ColmapQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:r"""Convert an Apple ARKit camera-to-world screen pose to the world-to-camera pose expected by Colmap.- Both poses in quaternion representation.+ ``q_colmap, t_colmap = ARKitQTVecs_to_ColmapQTVecs(q, t)`` takes ARKit camera-to-world poses in the graphics camera+ frame, as ``(w, x, y, z)`` quaternions ``(B, 4)`` and camera centres ``(B, 3, 1)``, and returns the Colmap+ world-to-camera poses in the vision camera frame, the ``QW QX QY QZ TX TY TZ`` of ``images.txt``, as a+ ``(B, 4)`` quaternion and a ``(B, 3, 1)`` translation. ARKit stores quaternions as ``(x, y, z, w)``: reorder them+ first. The sign of the returned quaternion is not canonical. For the reverse, use+ :func:`ColmapQTVecs_to_ARKitQTVecs`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the camera frames and@@ -3137,6 +3583,11 @@ def ARKitQTVecs_to_ColmapQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tupleqvec: Colmap rotation quaternion ``(B, 4)``, [w, x, y, z] format.tvec: translation vector ``(B, 3, 1)``, [x, y, z]+ See Also:+ - :func:`ColmapQTVecs_to_ARKitQTVecs`: the reverse conversion, Colmap to ARKit.+ - :func:`camtoworld_graphics_to_vision_Rt`: the camera-axes step of this pipeline, for a matrix pose.+ - :func:`camtoworld_to_worldtocam_Rt`: the inversion step of this pipeline, for a matrix pose.+Example:>>> q, t = torch.tensor([0, 1, 0, 1.])[None], torch.ones(3).reshape(1, 3, 1)>>> ARKitQTVecs_to_ColmapQTVecs(q, t)@@ -3158,7 +3609,11 @@ def ARKitQTVecs_to_ColmapQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tupledef ColmapQTVecs_to_ARKitQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:r"""Convert a Colmap world-to-camera pose to an Apple ARKit camera-to-world screen pose.- Both poses in quaternion representation.+ ``q_arkit, t_arkit = ColmapQTVecs_to_ARKitQTVecs(q, t)`` takes Colmap world-to-camera poses in the vision camera+ frame, as ``(w, x, y, z)`` quaternions ``(B, 4)`` and translations ``(B, 3, 1)``, and returns the ARKit+ camera-to-world poses in the graphics camera frame, as a ``(w, x, y, z)`` quaternion ``(B, 4)`` and the camera+ centre ``(B, 3, 1)``. Reorder the quaternion to ARKit's ``(x, y, z, w)`` before handing it over. The sign of the+ returned quaternion is not canonical. For the reverse, use :func:`ARKitQTVecs_to_ColmapQTVecs`.Convention:See :doc:`camera and world conventions </get-started/camera-conventions>` for the camera frames and@@ -3210,6 +3665,11 @@ def ColmapQTVecs_to_ARKitQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tupleqvec: ARKit rotation quaternion ``(B, 4)``, [w, x, y, z] format.tvec: translation vector ``(B, 3, 1)``, [x, y, z]+ See Also:+ - :func:`ARKitQTVecs_to_ColmapQTVecs`: the reverse conversion, ARKit to Colmap.+ - :func:`worldtocam_to_camtoworld_Rt`: the inversion step of this pipeline, for a matrix pose.+ - :func:`camtoworld_vision_to_graphics_Rt`: the camera-axes step of this pipeline, for a matrix pose.+Example:>>> q, t = torch.tensor([0, 1, 0, 1.])[None], torch.ones(3).reshape(1, 3, 1)>>> ColmapQTVecs_to_ARKitQTVecs(*ARKitQTVecs_to_ColmapQTVecs(q, t))@@ -3232,7 +3692,11 @@ def ColmapQTVecs_to_ARKitQTVecs(qvec: torch.Tensor, tvec: torch.Tensor) -> tupledef vector_to_skew_symmetric_matrix(vec: torch.Tensor) -> torch.Tensor:- r"""Convert a vector to a skew symmetric matrix.+ r"""Build the skew-symmetric cross-product matrix ``(3, 3)`` or ``(B, 3, 3)`` of a 3-vector.++ ``S = vector_to_skew_symmetric_matrix(v)`` takes a vector ``(3,)`` or a batch ``(B, 3)`` and returns the matrix+ ``(3, 3)`` or ``(B, 3, 3)`` with ``S @ x == cross(v, x)``. For any number of leading dimensions ``(*, 3)``, use+ :func:`~kornia.geometry.epipolar.cross_product_matrix`, which builds the same matrix.A vector :math:`(v1, v2, v3)` has a corresponding skew-symmetric matrix, which is of the form:@@ -3255,6 +3719,11 @@ def vector_to_skew_symmetric_matrix(vec: torch.Tensor) -> torch.Tensor:Returns:tensor of shape ``(3, 3)`` or ``(B, 3, 3)`` respectively.+ See Also:+ - :func:`~kornia.geometry.epipolar.cross_product_matrix`: the same matrix for ``(*, 3)`` vectors of any rank.+ - :meth:`~kornia.geometry.liegroup.So3.hat`: the same matrix, as the Lie-algebra element of a rotation+ vector.+Example:>>> vec = torch.tensor([1.0, 2.0, 3.0])>>> vector_to_skew_symmetric_matrix(vec)diff --git a/kornia/geometry/epipolar/__init__.py b/kornia/geometry/epipolar/__init__.pyindex ca80ba24f..ecf4ba603 100644--- a/kornia/geometry/epipolar/__init__.py+++ b/kornia/geometry/epipolar/__init__.py@@ -15,6 +15,45 @@# limitations under the License.#+r"""Two-view epipolar geometry: estimate F and E, recover the relative pose, triangulate.++The two-view functions take the first image's points first (``points1``, ``x1`` or ``pts1``), then the second+image's; the estimators take ``(B, N, 2)`` tensors. A fundamental matrix ``F`` relates pixel coordinates and an+essential matrix ``E`` normalized camera coordinates :math:`K^{-1} [u, v, 1]^\top`, both with+:math:`x_2^\top F x_1 = 0`: the second image's point on the left, as in OpenCV. Camera poses are world-to-camera+extrinsics, :math:`X_{cam} = R X + t`, and the relative motion ``(R, t)`` of camera 2 to camera 1 takes camera-1+coordinates to camera-2 coordinates, :math:`X_2 = R X_1 + t`; an essential matrix is :math:`[t]_\times R` of that+motion. :ref:`Two-view geometry <two-view-conventions>` compares these conventions with OpenCV's.++Which function to use:++- A model from correspondences known to be inliers: :func:`find_fundamental` from pixels (8 or 7 points),+ :func:`find_essential` from normalized camera coordinates (Nister's 5 points, up to ten candidates). For a plane+ or a pure rotation, :func:`~kornia.geometry.homography.find_homography_dlt`; with outliers,+ :class:`~kornia.geometry.ransac.RANSAC`.+- One model from another: :func:`essential_from_fundamental` and :func:`fundamental_from_essential` with the two+ cameras' intrinsics, :func:`essential_from_Rt` from two camera poses, :func:`fundamental_from_projections` from+ two projection matrices, and :func:`project_to_essential` onto the nearest essential matrix.+- The relative pose from an essential matrix: :func:`motion_from_essential_choose_solution` picks it by cheirality+ from pixel correspondences, and :func:`motion_from_essential` lists the four candidates;+ :func:`decompose_essential_matrix` (SVD) and :func:`decompose_essential_matrix_no_svd` (closed form) return the+ two rotations and the translation that they combine. The relative motion of two known camera poses:+ :func:`relative_camera_motion`.+- How well correspondences fit a fundamental matrix: :func:`sampson_epipolar_distance` (first-order geometric+ error), :func:`symmetrical_epipolar_distance` (both points to their epipolar lines),+ :func:`left_to_right_epipolar_distance` (in image 2) and :func:`right_to_left_epipolar_distance` (in image 1).+- Epipolar lines: :func:`compute_correspond_epilines`; the point on a line nearest to a correspondence,+ :func:`get_closest_point_on_epipolar_line`, built on :func:`get_perpendicular`.+- 3D points from two views: :func:`triangulate_points`.+- Cameras: :func:`projection_from_KRt` composes ``P = K [R | t]`` and :func:`KRt_from_projection` decomposes it;+ :func:`projections_from_fundamental` gives a canonical pair for ``F``; :func:`depth_from_point` gives the depth of+ points in a camera. Intrinsics: :func:`intrinsics_like` (from a focal length and an image size),+ :func:`random_intrinsics`, :func:`scale_intrinsics` (for a resized image).+- Helpers of the estimators: :func:`normalize_points` (Hartley normalization), :func:`normalize_transformation`+ (scale to a unit bottom-right entry), :func:`cross_product_matrix` (the matrix :math:`[x]_\times`).+- A random synthetic scene to test estimators on: :func:`generate_scene`.+"""+from ._metrics import (left_to_right_epipolar_distance,right_to_left_epipolar_distance,diff --git a/kornia/geometry/epipolar/_metrics.py b/kornia/geometry/epipolar/_metrics.pyindex 8035219ef..e84f7385a 100644--- a/kornia/geometry/epipolar/_metrics.py+++ b/kornia/geometry/epipolar/_metrics.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Module including useful metrics for Structure from Motion."""+"""Distances of correspondences to a fundamental matrix: Sampson, symmetric and one-sided."""import math@@ -32,10 +32,10 @@ def _sampson_epipolar_distance_manual_impl_(pts1: Tensor, pts2: Tensor, Fm: Tens"""Return Sampson distance for correspondences given the fundamental matrix.Args:- pts1: correspondences from the left images with shape :math:`(*, N, (2|3))`. If they are not homogeneous,- converted automatically.- pts2: correspondences from the right images with shape :math:`(*, N, (2|3))`. If they are not homogeneous,- converted automatically.+ pts1: correspondences from the left images with shape ``(*, N, 2)`` or ``(*, N, 3)``. If they are not+ homogeneous, converted automatically.+ pts2: correspondences from the right images with shape ``(*, N, 2)`` or ``(*, N, 3)``. If they are not+ homogeneous, converted automatically.Fm: Fundamental matrices with shape ``(*, 3, 3)``. Called Fm to avoid ambiguity with torch.nn.functional.squared: if True (default), the squared distance is returned.@@ -92,10 +92,10 @@ def _sampson_epipolar_distance_matmul_impl_(pts1: Tensor, pts2: Tensor, Fm: Tens"""Return Sampson distance for correspondences given the fundamental matrix.Args:- pts1: correspondences from the left images with shape :math:`(*, N, (2|3))`. If they are not homogeneous,- converted automatically.- pts2: correspondences from the right images with shape :math:`(*, N, (2|3))`. If they are not homogeneous,- converted automatically.+ pts1: correspondences from the left images with shape ``(*, N, 2)`` or ``(*, N, 3)``. If they are not+ homogeneous, converted automatically.+ pts2: correspondences from the right images with shape ``(*, N, 2)`` or ``(*, N, 3)``. If they are not+ homogeneous, converted automatically.Fm: Fundamental matrices with shape ``(*, 3, 3)``. Called Fm to avoid ambiguity with torch.nn.functional.squared: if True (default), the squared distance is returned.@@ -225,17 +225,26 @@ def sampson_epipolar_distance(eps: float = 1e-8,use_matmul_at_less_than_points: int = 10000,) -> Tensor:- r"""Return Sampson distance for correspondences given the fundamental matrix.+ r"""Compute the Sampson distance of correspondences to a fundamental matrix, to first order.++ ``d = sampson_epipolar_distance(pts1, pts2, Fm)`` takes matched points of the first and the second image,+ ``(*, N, 2)``, and fundamental matrices ``(*, 3, 3)`` with :math:`x_2^\top F x_1 = 0`, as+ :func:`find_fundamental` returns them, and returns the squared Sampson distance of each correspondence,+ ``(*, N)`` with the leading dimensions of the points and ``Fm`` broadcast. It approximates to first order the+ smallest squared distance by which the two points together must move to satisfy the epipolar constraint, in the+ squared units of the points; ``squared=False`` returns its square root. It is the residual that+ :class:`~kornia.geometry.ransac.RANSAC` scores fundamental and essential matrices with. For the distances of the+ points to the epipolar lines themselves, use :func:`symmetrical_epipolar_distance` or the one-sided+ :func:`left_to_right_epipolar_distance` and :func:`right_to_left_epipolar_distance`.Convention:- - ``pts1`` are first-image points, ``pts2`` second-image points, and ``Fm`` follows- :math:`x_2^\top F x_1 = 0`, as returned by :func:`find_fundamental`- (see :ref:`two-view geometry <two-view-conventions>`).- - Returns squared pixel distances by default; a 3-vector point is used as given, so it must have+ - ``pts1`` are first-image points and ``pts2`` second-image points+ (see :ref:`two-view geometry <two-view-conventions>`). A 3-vector point is used as given, so it must have:math:`w = 1`.- One set of correspondences scored against several matrices (points with leading dimensions of size 1)- is computed from two matrix products for all of them, in at least float32; the result matches the- per-matrix computation to roundoff.+ is computed from two matrix products for all of them, in at least float32: on the CPU always, and on+ other devices from ``2**16`` matrix-point pairs. The result matches the per-matrix computation to+ roundoff.- The value does not depend on the scale of ``Fm``, and ``squared=False`` returns exactly 0, with a zerogradient, for an exact match. A correspondence whose two epilines both have :math:`a = b = 0`, as under adegenerate ``Fm``, is at distance ``inf``.@@ -254,6 +263,14 @@ def sampson_epipolar_distance(Returns:the computed Sampson distance with shape ``(*, N)``.+ See Also:+ - :func:`symmetrical_epipolar_distance`: the summed squared distances of both points to their epipolar+ lines, not a first-order geometric error.+ - :func:`left_to_right_epipolar_distance`: the unsquared distance in image 2 only.+ - :func:`right_to_left_epipolar_distance`: the unsquared distance in image 1 only.+ - :func:`~kornia.geometry.homography.sampson_homography_distance`: the same first-order error for a+ homography.+"""# Checked here because the CUDA matmul and shared-point implementations do not check Fm. A single (3, 3) Fm# never counts as several models.@@ -344,11 +361,17 @@ def _symmetrical_epipolar_distance_matmul_impl_(pts1: Tensor, pts2: Tensor, Fm:def symmetrical_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor, squared: bool = True, eps: float = 1e-8) -> Tensor:- r"""Return symmetrical epipolar distance for correspondences given the fundamental matrix.+ r"""Compute the sum of the squared distances of both points of a correspondence to their epipolar lines.++ ``d = symmetrical_epipolar_distance(pts1, pts2, Fm)`` takes the arguments of :func:`sampson_epipolar_distance`+ and returns, per correspondence, ``(*, N)``, the squared distance of ``pts2`` to the epipolar line+ :math:`F x_1` in image 2 plus the squared distance of ``pts1`` to the line :math:`F^\top x_2` in image 1: the+ squares of :func:`left_to_right_epipolar_distance` and :func:`right_to_left_epipolar_distance` added.+ ``squared=False`` returns the square root of that sum. For the first-order geometric error of moving both points,+ use :func:`sampson_epipolar_distance`.Convention:- - Argument order and units as :func:`sampson_epipolar_distance`; the value is the sum of the two- squared point-to-epiline distances.+ - Argument order and units as :func:`sampson_epipolar_distance`.- The value does not depend on the scale of ``Fm``, and ``squared=False`` returns exactly 0 for an exactmatch, as in :func:`sampson_epipolar_distance`. A correspondence with an epiline of :math:`a = b = 0` ineither image is at distance ``inf``.@@ -363,6 +386,12 @@ def symmetrical_epipolar_distance(Returns:the computed Symmetrical distance with shape ``(*, N)``.+ See Also:+ - :func:`sampson_epipolar_distance`: the first-order geometric error, which RANSAC scores with.+ - :func:`left_to_right_epipolar_distance`: one of the two terms, unsquared: the distance in image 2.+ - :func:`right_to_left_epipolar_distance`: the other term, unsquared: the distance in image 1.+ - :func:`~kornia.geometry.homography.symmetric_transfer_error`: the two-image error of a homography.+"""# Checked here because the CUDA matmul implementation does not check Fm.if Fm.shape[-2:] != (3, 3):@@ -375,11 +404,17 @@ def symmetrical_epipolar_distance(def left_to_right_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor) -> Tensor:- r"""Return the one-sided distance from each point in ``pts2`` to the epipolar line of its match in ``pts1``.+ r"""Compute the distance in image 2 from each second-image point to the epipolar line of its match.++ ``d = left_to_right_epipolar_distance(pts1, pts2, Fm)`` takes the arguments of+ :func:`sampson_epipolar_distance` and returns, per correspondence, ``(*, N)``, the unsquared distance in the+ second image from ``pts2`` to the epipolar line :math:`F x_1` of its match in ``pts1``, in the units of the+ points. It does not measure anything in image 1: for the distance there, use+ :func:`right_to_left_epipolar_distance`, and for both images :func:`symmetrical_epipolar_distance` or+ :func:`sampson_epipolar_distance`.Convention:- - Argument order as :func:`sampson_epipolar_distance`; returns the unsquared pixel distance of each- ``pts2`` to the epipolar line of its ``pts1`` in the second image.+ - Argument order as :func:`sampson_epipolar_distance`.- The value does not depend on the scale of ``Fm``; a point whose epiline has :math:`a = b = 0` is atdistance ``inf``, as in :func:`~kornia.geometry.linalg.point_line_distance`.@@ -392,6 +427,12 @@ def left_to_right_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor) -> TReturns:the one-sided distance with shape ``(*, N)``.+ See Also:+ - :func:`right_to_left_epipolar_distance`: the distance in image 1 instead.+ - :func:`symmetrical_epipolar_distance`: both one-sided distances, squared and added.+ - :func:`sampson_epipolar_distance`: the first-order geometric error over both images.+ - :func:`~kornia.geometry.homography.oneway_transfer_error`: the image-2 error of a homography.+"""KORNIA_CHECK_IS_TENSOR(pts1)KORNIA_CHECK_IS_TENSOR(pts2)@@ -410,11 +451,17 @@ def left_to_right_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor) -> Tdef right_to_left_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor) -> Tensor:- r"""Return the one-sided distance from each point in ``pts1`` to the epipolar line of its match in ``pts2``.+ r"""Compute the distance in image 1 from each first-image point to the epipolar line of its match.++ ``d = right_to_left_epipolar_distance(pts1, pts2, Fm)`` takes the arguments of+ :func:`sampson_epipolar_distance` and returns, per correspondence, ``(*, N)``, the unsquared distance in the+ first image from ``pts1`` to the epipolar line :math:`F^\top x_2` of its match in ``pts2``, in the units of the+ points. It does not measure anything in image 2: for the distance there, use+ :func:`left_to_right_epipolar_distance`, and for both images :func:`symmetrical_epipolar_distance` or+ :func:`sampson_epipolar_distance`.Convention:- - Argument order as :func:`sampson_epipolar_distance`; returns the unsquared pixel distance of each- ``pts1`` to the epipolar line of its ``pts2`` in the first image.+ - Argument order as :func:`sampson_epipolar_distance`.- The value does not depend on the scale of ``Fm``; a point whose epiline has :math:`a = b = 0` is atdistance ``inf``, as in :func:`~kornia.geometry.linalg.point_line_distance`.@@ -427,6 +474,11 @@ def right_to_left_epipolar_distance(pts1: Tensor, pts2: Tensor, Fm: Tensor) -> TReturns:the one-sided distance with shape ``(*, N)``.+ See Also:+ - :func:`left_to_right_epipolar_distance`: the distance in image 2 instead.+ - :func:`symmetrical_epipolar_distance`: both one-sided distances, squared and added.+ - :func:`sampson_epipolar_distance`: the first-order geometric error over both images.+"""KORNIA_CHECK_IS_TENSOR(pts1)KORNIA_CHECK_IS_TENSOR(pts2)diff --git a/kornia/geometry/epipolar/essential.py b/kornia/geometry/epipolar/essential.pyindex e3bf537c4..6a9d7639a 100644--- a/kornia/geometry/epipolar/essential.py+++ b/kornia/geometry/epipolar/essential.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Module containing functionalities for the Essential matrix."""+"""Essential matrices: estimate them, convert from and to other models, recover the pose."""from typing import Any, Optional, Tuple, Union@@ -52,8 +52,8 @@ def run_5point(points1: torch.Tensor, points2: torch.Tensor, weights: Optional[tand the solver implemented referred to [@barath2020magsac++][@wei2023generalized][@wang2023vggsfm].Args:- points1: A set of calibrated points in the first image with a tensor shape :math:`(B, N, 2), N>=5`.- points2: A set of points in the second image with a tensor shape :math:`(B, N, 2), N>=5`.+ points1: A set of calibrated points in the first image with a tensor shape ``(B, N, 2)``, ``N >= 5``.+ points2: A set of points in the second image with a tensor shape ``(B, N, 2)``, ``N >= 5``.weights: Not used, kept for compatibility, including for :math:`N > 5`. Positive weights leave the nullspace of an exact five-point sample unchanged; for larger sets they can affect the least-squares fit.@@ -460,13 +460,20 @@ def null_to_Nister_solution(X: torch.Tensor, batch_size: int) -> torch.Tensor:def essential_from_fundamental(F_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) -> torch.Tensor:- r"""Get Essential matrix from Fundamental and Camera matrices.+ r"""Convert a fundamental matrix and the intrinsics of its two cameras to their essential matrix.- Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).+ ``E = essential_from_fundamental(F_mat, K1, K2)`` takes a fundamental matrix ``(*, 3, 3)`` in the+ :math:`x_2^\top F x_1 = 0` order of :func:`find_fundamental` and the pixel intrinsics ``(*, 3, 3)`` of the+ first and the second camera, and returns :math:`E = K_2^\top F K_1`, ``(*, 3, 3)`` with the leading dimensions+ broadcast, for which :math:`x_2^\top E x_1 = 0` in normalized camera coordinates (Hartley and Zisserman, 9.6,+ p. 257, eq. 9.12). It keeps the scale of ``F_mat`` and does not enforce the constraints of an essential matrix:+ pass an estimated one through :func:`project_to_essential` before decomposing it with+ :func:`decompose_essential_matrix_no_svd`; :func:`decompose_essential_matrix` does not need it. For the reverse, use+ :func:`fundamental_from_essential`.Convention:- - :math:`E = K_2^\top F K_1` with ``K1`` the camera of the first image, so ``E`` follows the- :math:`x_2^\top E x_1 = 0` order of :func:`find_essential`; it keeps the scale of ``F_mat``.+ - ``K1`` is the camera of the first image, so ``E`` follows the :math:`x_2^\top E x_1 = 0` order of+ :func:`find_essential`.Args:F_mat: The fundamental matrix with shape of ``(*, 3, 3)``.@@ -476,6 +483,11 @@ def essential_from_fundamental(F_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Returns:The essential matrix with shape ``(*, 3, 3)``.+ See Also:+ - :func:`fundamental_from_essential`: the reverse conversion, :math:`F = K_2^{-\top} E K_1^{-1}`.+ - :func:`project_to_essential`: the nearest matrix that satisfies the essential constraints.+ - :func:`find_essential`: an essential matrix fitted to calibrated correspondences directly.+"""KORNIA_CHECK_SHAPE(F_mat, ["*", "3", "3"])KORNIA_CHECK_SHAPE(K1, ["*", "3", "3"])@@ -484,13 +496,21 @@ def essential_from_fundamental(F_mat: torch.Tensor, K1: torch.Tensor, K2: torch.def project_to_essential(E_mat: torch.Tensor) -> torch.Tensor:- r"""Project a matrix onto the essential-matrix manifold.-- An essential matrix must be of the form :math:`U \text{diag}(s, s, 0) V^T`. Matrices estimated- by other means (e.g. a fundamental matrix fitted with the 8-point DLT) generally violate this- constraint, which silently breaks tools that rely on it, such as- :func:`decompose_essential_matrix` and :func:`motion_from_essential`. The projection averages- the two largest singular values and zeroes the smallest one.+ r"""Project 3x3 matrices onto the nearest essential matrices in Frobenius norm.++ ``E = project_to_essential(E_mat)`` takes matrices ``(*, 3, 3)`` and returns the closest matrices ``(*, 3, 3)``+ of the essential form :math:`U \text{diag}(s, s, 0) V^T`: the two largest singular values of ``E_mat`` averaged+ and the smallest one set to zero. Matrices estimated by other means, such as an 8-point fit of+ :func:`find_fundamental` on normalized camera coordinates or :func:`essential_from_fundamental` of an estimated+ ``F``, generally violate this constraint. :func:`decompose_essential_matrix_no_svd` then returns matrices that are+ not rotations, while :func:`decompose_essential_matrix` and :func:`motion_from_essential` return the same+ candidates for a matrix and for its projection, because the projection keeps its singular vectors. Their order can+ differ, though: on about half of noisy inputs the projection swaps the two rotations and flips the translation,+ so pick a candidate by its triangulated points, as :func:`motion_from_essential_choose_solution` does, not by its+ index. The candidates of :func:`find_essential` have this form only up to rounding, which on unlucky samples+ leaves a relative gap of order ``1e-4`` between their two largest singular values;+ :class:`~kornia.geometry.ransac.RANSAC` projects its essential matrix for that reason+ (`#3874 <https://github.com/kornia/kornia/issues/3874>`_).Convention:- The result is not normalised to unit Frobenius norm, unlike the candidates of :func:`find_essential`.@@ -503,6 +523,13 @@ def project_to_essential(E_mat: torch.Tensor) -> torch.Tensor:The closest matrices (in Frobenius norm) satisfying the essential-matrix constraint,with shape ``(*, 3, 3)``.+ See Also:+ - :func:`decompose_essential_matrix_no_svd`: the closed-form decomposition, which needs this projection of+ an estimate.+ - :func:`decompose_essential_matrix`: the SVD decomposition, which gives the same candidates with or without+ it.+ - :func:`essential_from_fundamental`: an essential matrix that usually needs this projection.+"""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])# A caller's autocast region would run the products below in half precision and return a half-precision matrix.@@ -523,14 +550,18 @@ def _project_to_essential(E_mat: torch.Tensor) -> torch.Tensor:def decompose_essential_matrix(E_mat: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:- r"""Decompose an essential matrix to possible rotations and translation.+ r"""Decompose an essential matrix into its two candidate rotations and its translation direction by SVD.- This function decomposes the essential matrix E using svd decomposition [96]- and give the possible solutions: :math:`R1, R2, t`.+ ``R1, R2, t = decompose_essential_matrix(E_mat)`` takes essential matrices ``(*, 3, 3)`` and returns the two+ rotations ``(*, 3, 3)`` and the unit translation ``(*, 3, 1)`` of their SVD factorization (Hartley and+ Zisserman, section 9.6.2): the relative motion of camera 2 to camera 1, :math:`X_2 = R X_1 + t` as in+ :func:`relative_camera_motion`, is one of :math:`(R_1, \pm t)` and :math:`(R_2, \pm t)`, with the translation+ known only up to scale. It does not tell which; :func:`motion_from_essential` stacks the four candidates and+ :func:`motion_from_essential_choose_solution` picks the one that puts the points in front of both cameras. For+ the same candidates in closed form, without an SVD, use :func:`decompose_essential_matrix_no_svd`.Convention:- - Returns two rotations and a unit translation; the true pose is one of :math:`(R_1, \pm t)`,- :math:`(R_2, \pm t)`. :math:`(R_1, -t)` and :math:`(R_2, t)` give :math:`[t]_\times R` with the sign+ - :math:`(R_1, -t)` and :math:`(R_2, t)` give :math:`[t]_\times R` with the signof ``E_mat``, :math:`(R_1, t)` and :math:`(R_2, -t)` the opposite sign. Which of the four is the truepose is not fixed: it can change when ``E_mat`` is negated or rescaled. Select by cheirality with:func:`motion_from_essential_choose_solution`; :ref:`two-view geometry <two-view-conventions>`@@ -543,7 +574,14 @@ def decompose_essential_matrix(E_mat: torch.Tensor) -> Tuple[torch.Tensor, torchReturns:A tuple containing the first and second possible rotation matrices and the translation vector,- with shapes :math:`[(*, 3, 3), (*, 3, 3), (*, 3, 1)]`: the same leading dims as the input.+ with shapes ``(*, 3, 3)``, ``(*, 3, 3)`` and ``(*, 3, 1)``: the same leading dims as the input.++ See Also:+ - :func:`decompose_essential_matrix_no_svd`: the same candidates in closed form, exact essential+ matrices only, with the leading dims flattened.+ - :func:`motion_from_essential`: the four candidate poses stacked on one axis.+ - :func:`motion_from_essential_choose_solution`: the one candidate with the points in front of both+ cameras."""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])@@ -584,14 +622,20 @@ def _decompose_essential_matrix(E_mat: torch.Tensor) -> Tuple[torch.Tensor, torcdef decompose_essential_matrix_no_svd(E_mat: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:- r"""Decompose the essential matrix to rotation and translation.+ r"""Decompose an essential matrix into candidate rotations and translation in closed form.- Recovers the rotations and translation from an essential matrix without SVD.- Reference: Horn, Berthold KP. Recovering baseline and orientation from essential matrix[J].- J. Opt. Soc. Am, 1990, 110.+ ``R1, R2, t = decompose_essential_matrix_no_svd(E_mat)`` takes essential matrices ``(*, 3, 3)`` and returns the+ candidates of :func:`decompose_essential_matrix`, two rotations and a unit translation, from cross products of+ the columns of ``E_mat`` and its cofactor matrix (Horn, Recovering baseline and orientation from essential+ matrix, J. Opt. Soc. Am., 1990). Every leading dimension is flattened into one batch dimension ``B``, so a+ ``(3, 3)`` input returns a batch of one. The closed form holds only for an exact essential matrix: for one off+ the essential manifold, such as an estimate, the returned matrices are in general not rotations, so pass it+ through :func:`project_to_essential` first or use :func:`decompose_essential_matrix`, whose rotations are+ orthogonal for any input.Convention:- - Same candidate set as :func:`decompose_essential_matrix`, with ``t`` of unit norm.+ - For an exact essential matrix, the same candidate set as :func:`decompose_essential_matrix`, with ``t``+ of unit norm.- A caller's autocast region does not change the result, outside TorchScript.Args:@@ -599,7 +643,13 @@ def decompose_essential_matrix_no_svd(E_mat: torch.Tensor) -> Tuple[torch.TensorReturns:A tuple containing the first and second possible rotation matrices and the translation vector, with- shapes :math:`[(B, 3, 3), (B, 3, 3), (B, 3, 1)]`: all leading dims are flattened into one batch dim.+ shapes ``(B, 3, 3)``, ``(B, 3, 3)`` and ``(B, 3, 1)``: all leading dims are flattened into one batch dim.++ See Also:+ - :func:`decompose_essential_matrix`: the same candidates by SVD, with orthogonal rotations for any input+ and the leading dims kept.+ - :func:`motion_from_essential`: the four candidate poses stacked on one axis.+ - :func:`project_to_essential`: the nearest exact essential matrix, to decompose an estimate."""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])@@ -664,14 +714,15 @@ def _decompose_essential_matrix_no_svd(E_mat: torch.Tensor) -> Tuple[torch.Tensodef essential_from_Rt(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) -> torch.Tensor:- r"""Get the Essential matrix from Camera motion (Rs and ts).-- Reference: Hartley/Zisserman 9.6 pag 257 (formula 9.12)+ r"""Build the essential matrix of two cameras from their world-to-camera rotations and translations.- Convention:- - Returns :math:`[t]_\times R` for the relative motion ``(R, t)`` of :func:`relative_camera_motion`, so- :math:`x_2^\top E x_1 = 0` in normalised camera coordinates. It is not normalised:- :math:`\|E\|_F = \sqrt{2} \|t\|`.+ ``E = essential_from_Rt(R1, t1, R2, t2)`` takes the world-to-camera extrinsics of the first and the second+ camera, :math:`X_{cam} = R X + t`, as rotations ``(*, 3, 3)`` and translations ``(*, 3, 1)``, and returns+ :math:`E = [t]_\times R`, ``(*, 3, 3)``, for their relative motion ``(R, t)`` from :func:`relative_camera_motion`,+ so that :math:`x_2^\top E x_1 = 0` in normalized camera coordinates (Hartley and Zisserman, 9.6, p. 257,+ eq. 9.12). It is not normalized: :math:`\|E\|_F = \sqrt{2} \|t\|`, unlike the unit-norm candidates of+ :func:`find_essential`. For the fundamental matrix of two projection matrices, use+ :func:`fundamental_from_projections`.Args:R1: The first camera rotation matrix with shape ``(*, 3, 3)``.@@ -682,6 +733,11 @@ def essential_from_Rt(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2:Returns:The Essential matrix with the shape ``(*, 3, 3)``.+ See Also:+ - :func:`relative_camera_motion`: the relative motion ``(R, t)`` that this matrix encodes.+ - :func:`fundamental_from_projections`: the uncalibrated counterpart, from two projection matrices.+ - :func:`decompose_essential_matrix`: the reverse, the motion candidates of an essential matrix.+"""KORNIA_CHECK_SHAPE(R1, ["*", "3", "3"])KORNIA_CHECK_SHAPE(R2, ["*", "3", "3"])@@ -698,22 +754,29 @@ def essential_from_Rt(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2:def motion_from_essential(E_mat: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:- r"""Get Motion (R's and t's ) from Essential matrix.+ r"""List the four candidate relative poses of an essential matrix, stacked on one axis.- Computes and return four possible poses exist for the decomposition of the Essential- matrix. The possible solutions are :math:`[R1,t], [R1,-t], [R2,t], [R2,-t]`.+ ``Rs, ts = motion_from_essential(E_mat)`` takes essential matrices ``(*, 3, 3)`` and returns the four candidate+ motions of camera 2 relative to camera 1 that the matrix allows, rotations ``(*, 4, 3, 3)`` and unit translations+ ``(*, 4, 3, 1)``, in the order :math:`(R_1, t), (R_1, -t), (R_2, t), (R_2, -t)` of the candidates of+ :func:`decompose_essential_matrix`. Only one of them puts the scene in front of both cameras; to select it from+ point correspondences, use :func:`motion_from_essential_choose_solution`.Convention:- - The candidates of :func:`decompose_essential_matrix` are stacked on dim ``-3`` in that order; which- index is the true pose is not fixed.+ - The candidates are stacked on dim ``-3``; which index is the true pose is not fixed.- A caller's autocast region does not change the result, as for :func:`decompose_essential_matrix`.Args:E_mat: The essential matrix in the form of ``(*, 3, 3)``.Returns:- The rotation and translation containing the four possible combination for the retrieved motion.- The tuple is as following :math:`[(*, 4, 3, 3), (*, 4, 3, 1)]`.+ The rotation and translation containing the four possible combination for the retrieved motion,+ with shapes ``(*, 4, 3, 3)`` and ``(*, 4, 3, 1)``.++ See Also:+ - :func:`motion_from_essential_choose_solution`: the one candidate that passes the cheirality test.+ - :func:`decompose_essential_matrix`: the two rotations and the translation the candidates combine.+ - :func:`decompose_essential_matrix_no_svd`: the same rotations and translation without an SVD."""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])@@ -736,20 +799,23 @@ def motion_from_essential_choose_solution(x2: torch.Tensor,mask: Optional[torch.Tensor] = None,) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:- r"""Recover the relative camera rotation and the translation from an estimated essential matrix.-- The method checks the corresponding points in two images and also returns the triangulated- 3d points. Internally uses :py:meth:`~kornia.geometry.epipolar.decompose_essential_matrix` and- :py:meth:`~kornia.geometry.epipolar.triangulate_points`.+ r"""Pick the relative pose of an essential matrix that puts the points in front of both cameras.++ ``R, t, points3d, count = motion_from_essential_choose_solution(E_mat, K1, K2, x1, x2)`` takes essential matrices+ ``(B, 3, 3)`` with :math:`x_2^\top E x_1 = 0` in normalized camera coordinates, the pixel intrinsics+ ``(B, 3, 3)`` of the first and the second camera, and pixel correspondences ``(B, N, 2)`` of the first and the+ second image. Of the four candidates of :func:`motion_from_essential` it keeps the one that puts the most points+ at positive depth in both cameras. It returns that candidate's rotation ``(B, 3, 3)`` and unit translation+ ``(B, 3, 1)`` of camera 2 relative to camera 1, :math:`X_2 = R X_1 + t` as in :func:`relative_camera_motion`. It+ also returns the points triangulated by :func:`triangulate_points` in the frame of camera 1 at the scale+ :math:`\|t\| = 1`, ``(B, N, 3)``, and the number of points that passed, ``(B,)``, the count that+ ``cv2.recoverPose`` also returns. For all four candidates, use :func:`motion_from_essential`.Convention:- ``K1`` and ``K2`` are applied inside, so ``x1`` and ``x2`` are pixel coordinates.- - Returns the candidate with the most points at positive depth in both cameras, with- :math:`\|t\| = 1`, and the points triangulated in the first camera's frame at that scale.- - The fourth output is the number of points at positive depth in both cameras for the returned- candidate, as ``cv2.recoverPose`` returns. A count of ``0`` means no point passed, and the returned- pose is then candidate 0 of- :py:meth:`~kornia.geometry.epipolar.motion_from_essential`, not a recovered motion.+ - The count is the number of points at positive depth in both cameras for the returned+ candidate. A count of ``0`` means no point passed, and the returned pose is then candidate 0 of+ :func:`motion_from_essential`, not a recovered motion.- A caller's autocast region does not change the result. ``K1`` and ``K2`` are cast to the dtype of``E_mat``.@@ -768,9 +834,15 @@ def motion_from_essential_choose_solution(Returns:The rotation, translation, 3d triangulated points, and the number of pointswith positive depth in both cameras for the selected candidate.- The tuple is as following :math:`[(B, 3, 3), (B, 3, 1), (B, N, 3), (B,)]`,+ The tuple is as following ``(B, 3, 3)``, ``(B, 3, 1)``, ``(B, N, 3)`` and ``(B,)``,without ``B`` for unbatched inputs.+ See Also:+ - :func:`motion_from_essential`: all four candidate poses, without a cheirality test.+ - :func:`decompose_essential_matrix`: the rotations and translation the candidates combine.+ - :func:`triangulate_points`: the triangulation it tests the candidates with.+ - :func:`find_essential`: the essential matrices to pass, from normalized camera coordinates.+"""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])KORNIA_CHECK_SHAPE(K1, ["*", "3", "3"])@@ -856,11 +928,15 @@ def motion_from_essential_choose_solution(def relative_camera_motion(R1: torch.Tensor, t1: torch.Tensor, R2: torch.Tensor, t2: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:- r"""Compute the relative camera motion between two cameras.+ r"""Compute the motion from camera 1 to camera 2 from their world-to-camera rotations and translations.- Given the motion parameters of two cameras, computes the motion parameters of the second- one assuming the first one to be at the origin. If :math:`T1` and :math:`T2` are the camera motions,- the computed relative motion is :math:`T = T_{2}T^{-1}_{1}`.+ ``R, t = relative_camera_motion(R1, t1, R2, t2)`` takes the world-to-camera extrinsics of two cameras,+ rotations ``(*, 3, 3)`` and translations ``(*, 3, 1)``, and returns the rotation ``(*, 3, 3)`` and translation+ ``(*, 3, 1)`` that take camera-1 coordinates to camera-2 coordinates, :math:`X_2 = R X_1 + t`: the pose of the+ second camera when the first one is at the origin. This is the motion that :func:`essential_from_Rt` encodes and+ :func:`motion_from_essential_choose_solution` recovers. For the same motion from ``4 x 4`` extrinsics, compute+ ``E2 @ inverse_transformation(E1)`` with :func:`~kornia.geometry.linalg.inverse_transformation`;+ :func:`~kornia.geometry.linalg.relative_transformation` composes in the other order.Convention:- Inputs are world-to-camera extrinsics, :math:`x_{cam} = R X + t` (see :doc:`camera and world@@ -877,7 +953,13 @@ def relative_camera_motion(Returns:A tuple with the relative rotation matrix and- translation vector with the shape of :math:`[(*, 3, 3), (*, 3, 1)]`.+ translation vector with the shapes ``(*, 3, 3)`` and ``(*, 3, 1)``.++ See Also:+ - :func:`essential_from_Rt`: the essential matrix of this motion.+ - :func:`~kornia.geometry.linalg.relative_transformation`: the relative pose of two 4x4 matrices,+ :math:`E_1^{-1} E_2` rather than :math:`E_2 E_1^{-1}`.+ - :func:`motion_from_essential_choose_solution`: the same motion, up to scale, from an essential matrix."""KORNIA_CHECK_SHAPE(R1, ["*", "3", "3"])@@ -897,14 +979,23 @@ def relative_camera_motion(def find_essential(points1: torch.Tensor, points2: torch.Tensor, weights: Optional[torch.Tensor] = None) -> torch.Tensor:- r"""Find essential matrices.+ r"""Estimate up to ten essential matrices from calibrated points with Nister's five-point solver.++ ``Es = find_essential(points1, points2)`` takes correspondences ``(B, N, 2)``, ``N >= 5``, of the first and the+ second image in normalized camera coordinates :math:`K^{-1} [u, v, 1]^\top`, not pixels (see+ :func:`~kornia.geometry.conversions.normalize_points_with_intrinsics`), and returns ``(B, 10, 3, 3)``: one slot+ per root of Nister's degree-ten polynomial, a candidate of unit Frobenius norm for each real root and ``NaN`` for+ each complex one. Five points satisfy :math:`x_2^\top E x_1 = 0` exactly for every candidate; with more, the null+ space of the epipolar constraints is fitted by least squares first, so the candidates satisfy it only+ approximately. Pick a candidate by its :func:`sampson_epipolar_distance` on other+ correspondences, ignoring the ``NaN`` slots. For one essential matrix from correspondences with outliers, use+ :class:`~kornia.geometry.ransac.RANSAC` with ``model_type="essential"``; for pixel correspondences of unknown+ cameras, :func:`find_fundamental`.Convention:- - ``points1`` (first image) and ``points2`` (second image) are normalised camera coordinates- :math:`K^{-1} [u, v, 1]^\top`, not pixels; each real candidate satisfies :math:`x_2^\top E x_1 = 0`- in them. :ref:`Two-view geometry <two-view-conventions>` maps this onto OpenCV.- - All ten slots are always returned: each real root gives a candidate of unit Frobenius norm, and each- complex root a ``NaN`` slot, so a sample with no real solution returns ten ``NaN`` slots.+ - :ref:`Two-view geometry <two-view-conventions>` maps the argument order and the candidate layout onto+ OpenCV.+ - All ten slots are always returned, so a sample with no real solution returns ten ``NaN`` slots.- The solve runs in float64 whatever the input dtype, and the degree-ten polynomial's roots come from LAPACKon the host for every device. MPS has no float64, so MPS inputs are solved on the host and the candidatescopied back, which on an Apple M1 was also faster than a float32 solve on the device. A sample whose five@@ -912,8 +1003,8 @@ def find_essential(``NaN`` slots and a zero gradient.Args:- points1: A set of points in the first image with a tensor shape :math:`(B, N, 2), N>=5`.- points2: A set of points in the second image with a tensor shape :math:`(B, N, 2), N>=5`.+ points1: A set of points in the first image with a tensor shape ``(B, N, 2)``, ``N >= 5``.+ points2: A set of points in the second image with a tensor shape ``(B, N, 2)``, ``N >= 5``.weights: Accepted with a shape of ``(B, N)`` and ignored for compatibility, including for :math:`N > 5`.Positive weights leave the null space of an exact five-point sample unchanged; for larger sets they canaffect the least-squares fit.@@ -923,6 +1014,14 @@ def find_essential(To choose the best one out of 10, try to check the one with the lowest Sampson distance, ignoring the``NaN`` slots.+ See Also:+ - :func:`find_fundamental`: the model of uncalibrated cameras, from pixel correspondences.+ - :func:`~kornia.geometry.homography.find_homography_dlt`: a homography instead, for a planar scene or a+ camera that only rotates.+ - :class:`~kornia.geometry.ransac.RANSAC`: one essential matrix, robust to wrong correspondences.+ - :func:`motion_from_essential_choose_solution`: the relative pose of a chosen candidate.+ - :func:`essential_from_fundamental`: an essential matrix from a fundamental matrix and both intrinsics.+"""return run_5point(points1, points2, weights).to(points1.dtype)diff --git a/kornia/geometry/epipolar/fundamental.py b/kornia/geometry/epipolar/fundamental.pyindex aaa6502d3..514ca034c 100644--- a/kornia/geometry/epipolar/fundamental.py+++ b/kornia/geometry/epipolar/fundamental.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Module containing the functionalities for computing the Fundamental Matrix."""+"""Fundamental matrices: estimate them, convert from other models, work with epipolar lines."""import mathfrom typing import Literal, Optional, Tuple, Union@@ -32,13 +32,16 @@ from kornia.geometry.solvers.polynomial_solver import _solve_cubic_realdef normalize_points(points: torch.Tensor, eps: float = 1e-8, weights: Optional[torch.Tensor] = None) -> Tuple[torch.Tensor, torch.Tensor]:- r"""Normalize points (isotropic).+ r"""Normalize 2D point sets to zero mean and a mean distance of sqrt(2) from the origin, Hartley's way.- Computes the Hartley normalisation: the points are translated to zero mean and scaled isotropically so- that their mean distance to the origin is :math:`\sqrt{2}`. Reference: Hartley/Zisserman 4.4.4 pag.107-- This operation is an essential step before applying the DLT algorithm in order to consider- the result as optimal.+ ``points_norm, T = normalize_points(points)`` takes point sets ``(B, N, 2)`` and returns them translated to zero+ mean and scaled isotropically so that their mean distance to the origin is :math:`\sqrt{2}`, ``(B, N, 2)``,+ together with the similarity ``T`` ``(B, 3, 3)`` that maps the input points to them (Hartley and Zisserman,+ 4.4.4, p. 107). This is an essential step before the DLT, which is otherwise badly conditioned for pixel+ coordinates: :func:`find_fundamental` and :func:`~kornia.geometry.homography.find_homography_dlt` normalize both+ point sets with it and undo it on their result, as :math:`T_2^\top F T_1` and :math:`T_2^{-1} H T_1`. With+ ``weights`` the mean and the mean distance are weighted. For the :math:`[-1, 1]` normalization by the image size,+ use :func:`~kornia.geometry.conversions.normalize_pixel_coordinates`.Args:points: Tensor containing the points to be normalized with shape ``(B, N, 2)``.@@ -50,6 +53,11 @@ def normalize_points(tuple containing the normalized points in the shape ``(B, N, 2)`` and the transformation matrixin the shape ``(B, 3, 3)`` that maps the input points to them.+ See Also:+ - :func:`normalize_transformation`: the other scaling step of the estimators, a unit bottom-right entry.+ - :func:`~kornia.geometry.conversions.normalize_pixel_coordinates`: a fixed normalization to+ :math:`[-1, 1]` by the image size, not by the point statistics.+"""if points.ndim != 3:raise AssertionError(points.shape)@@ -107,11 +115,15 @@ def normalize_points(def normalize_transformation(M: torch.Tensor, eps: float = 1e-8) -> torch.Tensor:- r"""Normalize a given transformation matrix.+ r"""Scale matrices so that their bottom-right entry is 1, the gauge of a homography or F.++ ``M_norm = normalize_transformation(M)`` takes matrices ``(*, m, n)`` and divides each by its last entry+ ``M[..., -1, -1]``, returning the same shape. This gives :func:`find_fundamental` its ``F[2, 2] = 1`` scaling,+ and :func:`~kornia.geometry.homography.find_homography_dlt` its ``H[2, 2] = 1``. For the normalization of point+ sets before such a fit, use :func:`normalize_points`.Convention:- - Divides ``M`` by its last entry ``M[..., -1, -1]``, which gives :func:`find_fundamental` its- ``F[2, 2] = 1`` scaling. A matrix whose last entry is within ``eps`` of zero is returned unchanged.+ - A matrix whose last entry is within ``eps`` of zero is returned unchanged, at its own scale.Args:M: The transformation to be normalized of any shape with a minimum size of 2x2.@@ -120,6 +132,9 @@ def normalize_transformation(M: torch.Tensor, eps: float = 1e-8) -> torch.TensorReturns:the normalized transformation matrix with same shape as the input.+ See Also:+ - :func:`normalize_points`: the Hartley normalization of the points the estimators fit.+"""if len(M.shape) < 2:raise AssertionError(M.shape)@@ -595,12 +610,19 @@ def find_fundamental(weights: Optional[torch.Tensor] = None,method: Literal["8POINT", "7POINT"] = "8POINT",) -> torch.Tensor:- r"""Find the fundamental matrix.+ r"""Estimate the fundamental matrix from pixel correspondences with the 8- or 7-point algorithm.++ ``F = find_fundamental(points1, points2)`` takes correspondences ``(B, N, 2)``, ``N >= 8``, of the first and the+ second image, in pixels or any other image coordinates, and returns the rank-2 matrix ``(B, 3, 3)`` with+ :math:`x_2^\top F x_1 = 0` (the second image's point on the left) of the Hartley-normalized, optionally weighted,+ 8-point least-squares fit. ``method="7POINT"`` takes exactly seven correspondences and returns the three+ candidates ``(B, 3, 3, 3)`` of the 7-point algorithm instead. For calibrated cameras and normalized camera+ coordinates, use :func:`find_essential`; for a planar scene or a camera that only rotates, where ``F`` is not+ determined, :func:`~kornia.geometry.homography.find_homography_dlt`; with outliers,+ :class:`~kornia.geometry.ransac.RANSAC`.Convention:- - ``points1`` are in the first image and ``points2`` in the second; the result satisfies- :math:`x_2^\top F x_1 = 0`, the second image's point on the left.- :ref:`Two-view geometry <two-view-conventions>` maps this onto OpenCV.+ - :ref:`Two-view geometry <two-view-conventions>` maps the argument order onto OpenCV.- The result is scaled so that ``F[2, 2] = 1`` by :func:`normalize_transformation`, which leaves it at itsunnormalised scale when ``F[2, 2]`` is numerically zero, as for exactly rectified stereo.``method="7POINT"`` returns three candidates in no particular order.@@ -623,6 +645,14 @@ def find_fundamental(Raises:ValueError: If an invalid method is provided.+ See Also:+ - :func:`find_essential`: the model of calibrated cameras, from normalized camera coordinates.+ - :func:`~kornia.geometry.homography.find_homography_dlt`: a homography, the model of a plane or of a+ rotating camera.+ - :class:`~kornia.geometry.ransac.RANSAC`: one fundamental matrix, robust to wrong correspondences.+ - :func:`sampson_epipolar_distance`: the residual of correspondences under the result.+ - :func:`fundamental_from_essential`: a fundamental matrix from an essential matrix and both intrinsics.+"""if method.upper() == "7POINT":result = run_7point(points1, points2)@@ -634,11 +664,18 @@ def find_fundamental(def compute_correspond_epilines(points: torch.Tensor, F_mat: torch.Tensor) -> torch.Tensor:- r"""Compute the corresponding epipolar line for a given set of points.+ r"""Compute the epipolar lines of points in the other image, scaled to a unit normal.++ ``lines = compute_correspond_epilines(points, F_mat)`` takes points ``(*, N, 2)``, or homogeneous+ ``(*, N, 3)``, and fundamental matrices ``(*, 3, 3)``, and returns the lines :math:`F x` as ``(a, b, c)`` with+ :math:`ax + by + c = 0`, ``(*, N, 3)``, scaled to :math:`a^2 + b^2 = 1` so that :math:`|ax + by + c|` is the+ distance of a point to them. For first-image points and an ``F`` from :func:`find_fundamental` the lines lie in+ the second image; for second-image points pass ``F.transpose(-2, -1)``. For the distance of the matching points+ to these lines, use :func:`left_to_right_epipolar_distance`.Convention:- - For first-image points and ``F`` from :func:`find_fundamental` the lines lie in the second image;- for second-image points pass ``F.transpose(-2, -1)``. Lines are scaled to :math:`a^2 + b^2 = 1`.+ - As OpenCV's ``computeCorrespondEpilines`` with ``whichImage=1``; see+ :ref:`two-view geometry <two-view-conventions>`. A line with :math:`a = b = 0` is returned unscaled.Args:points: tensor containing the set of points to project in the shape of ``(*, N, 2)`` or ``(*, N, 3)``.@@ -649,6 +686,10 @@ def compute_correspond_epilines(points: torch.Tensor, F_mat: torch.Tensor) -> toEach line is described as:math:`ax + by + c = 0` and encoding the vectors as :math:`(a, b, c)`.+ See Also:+ - :func:`left_to_right_epipolar_distance`: the distance of each second-image point to these lines.+ - :func:`get_closest_point_on_epipolar_line`: the point of each line nearest to the matching point.+"""KORNIA_CHECK_SHAPE(points, ["*", "N", "DIM"])if points.shape[-1] == 2:@@ -674,7 +715,13 @@ def compute_correspond_epilines(points: torch.Tensor, F_mat: torch.Tensor) -> todef get_perpendicular(lines: torch.Tensor, points: torch.Tensor) -> torch.Tensor:- r"""Compute the perpendicular to a line, through the point.+ r"""Compute the line through each point that is perpendicular to a given line.++ ``perp = get_perpendicular(lines, points)`` takes lines ``(B, N, 3)`` as ``(a, b, c)`` with+ :math:`ax + by + c = 0` and points ``(B, N, 2)``, or homogeneous ``(B, N, 3)``, paired by index, and returns the+ line ``(B, N, 3)`` through each point perpendicular to its line, :math:`(-b, a, bx - ay)` for the point+ :math:`(x, y)`. :func:`get_closest_point_on_epipolar_line` intersects it with the epipolar line; for that point+ directly, use that function.Args:lines: tensor containing the set of lines ``(B, N, 3)``.@@ -686,6 +733,10 @@ def get_perpendicular(lines: torch.Tensor, points: torch.Tensor) -> torch.Tensor:math:`ax + by + c = 0` and encoding the vectors as :math:`(a, b, c)`; the normal :math:`(a, b)` has thenorm of the input line's and is not rescaled.+ See Also:+ - :func:`get_closest_point_on_epipolar_line`: the foot of this perpendicular on an epipolar line.+ - :func:`compute_correspond_epilines`: the epipolar lines to pass as ``lines``.+"""KORNIA_CHECK_SHAPE(lines, ["*", "N", "3"])KORNIA_CHECK_SHAPE(points, ["*", "N", "two"])@@ -701,11 +752,15 @@ def get_perpendicular(lines: torch.Tensor, points: torch.Tensor) -> torch.Tensordef get_closest_point_on_epipolar_line(pts1: torch.Tensor, pts2: torch.Tensor, Fm: torch.Tensor) -> torch.Tensor:- r"""Return closest point on the epipolar line to the correspondence, given the fundamental matrix.+ r"""Move each second-image point to the nearest point of the epipolar line of its first-image match.++ ``points = get_closest_point_on_epipolar_line(pts1, pts2, Fm)`` takes first- and second-image points+ ``(B, N, 2)``, or homogeneous ``(B, N, 3)``, and fundamental matrices ``(B, 3, 3)`` in the+ :math:`x_2^\top F x_1 = 0` order of :func:`find_fundamental`, and returns, in the second image, the orthogonal+ projection ``(B, N, 2)`` of each ``pts2`` onto the epipolar line :math:`F x_1` of its ``pts1``. Only the+ second-image points move. For the distance between the two, use :func:`left_to_right_epipolar_distance`.Convention:- - Returns, in the second image, the point of the epipolar line of ``pts1`` closest to ``pts2``, for- ``Fm`` in the :math:`x_2^\top F x_1 = 0` order of :func:`find_fundamental`.- ``Fm`` and ``pts2`` take the dtype of ``pts1``. A caller's autocast region does not change the result,outside TorchScript.@@ -719,6 +774,11 @@ def get_closest_point_on_epipolar_line(pts1: torch.Tensor, pts2: torch.Tensor, FReturns:point on epipolar line ``(B, N, 2)``.+ See Also:+ - :func:`left_to_right_epipolar_distance`: the distance from each ``pts2`` to the returned point.+ - :func:`compute_correspond_epilines`: the epipolar lines the points are projected onto.+ - :func:`get_perpendicular`: the perpendicular through ``pts2`` that this intersects with the line.+"""if not isinstance(Fm, torch.Tensor):raise TypeError(f"Fm type is not a torch.Tensor. Got {type(Fm)}")@@ -748,14 +808,18 @@ def _get_closest_point_on_epipolar_line(pts1: torch.Tensor, pts2: torch.Tensor,def fundamental_from_essential(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Tensor) -> torch.Tensor:- r"""Get the Fundamental matrix from Essential and camera matrices.+ r"""Convert an essential matrix and the intrinsics of its two cameras to their fundamental matrix.- Uses the method from Hartley/Zisserman 9.6 pag 257 (formula 9.12).+ ``F = fundamental_from_essential(E_mat, K1, K2)`` takes an essential matrix ``(*, 3, 3)`` with+ :math:`x_2^\top E x_1 = 0` in normalized camera coordinates and the pixel intrinsics ``(*, 3, 3)`` of the+ first and the second camera, all with the same number of dimensions, and returns+ :math:`F = K_2^{-\top} E K_1^{-1}`, ``(*, 3, 3)``, for which :math:`x_2^\top F x_1 = 0` in pixels (Hartley and+ Zisserman, 9.6, p. 257, eq. 9.12). A batch element with a singular camera matrix is NaN. For the reverse, use+ :func:`essential_from_fundamental`.Convention:- - :math:`F = K_2^{-\top} E K_1^{-1}` with ``K1`` the camera of the first image, so ``F`` follows the- :math:`x_2^\top F x_1 = 0` order of :func:`find_fundamental`; it keeps the scale of ``E_mat``, with no- ``F[2, 2] = 1`` normalisation.+ - ``K1`` is the camera of the first image, so ``F`` follows the :math:`x_2^\top F x_1 = 0` order of+ :func:`find_fundamental`; it keeps the scale of ``E_mat``, with no ``F[2, 2] = 1`` normalisation.Args:E_mat: The essential matrix with shape of ``(*, 3, 3)``.@@ -765,6 +829,11 @@ def fundamental_from_essential(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.Returns:The fundamental matrix with shape ``(*, 3, 3)``.+ See Also:+ - :func:`essential_from_fundamental`: the reverse conversion, :math:`E = K_2^\top F K_1`.+ - :func:`fundamental_from_projections`: a fundamental matrix from two projection matrices instead.+ - :func:`find_fundamental`: a fundamental matrix fitted to pixel correspondences directly.+"""KORNIA_CHECK_SHAPE(E_mat, ["*", "3", "3"])KORNIA_CHECK_SHAPE(K1, ["*", "3", "3"])@@ -787,12 +856,18 @@ def fundamental_from_essential(E_mat: torch.Tensor, K1: torch.Tensor, K2: torch.def fundamental_from_projections(P1: torch.Tensor, P2: torch.Tensor) -> torch.Tensor:- r"""Get the Fundamental matrix from Projection matrices.+ r"""Compute the fundamental matrix of two cameras from their 3x4 projection matrices.++ ``F = fundamental_from_projections(P1, P2)`` takes the projection matrices ``(*, 3, 4)`` of the first and the+ second camera, with the same leading dimensions, and returns their fundamental matrix ``(*, 3, 3)``, for which+ :math:`x_2^\top F x_1 = 0` in the image coordinates of ``P1`` and ``P2``: the order of :func:`find_fundamental`.+ Each entry is a 4x4 determinant of two rows of each matrix. For a canonical camera pair of a fundamental matrix,+ the other way round, use :func:`projections_from_fundamental`; for calibrated cameras given by their poses,+ :func:`essential_from_Rt`.Convention:- - The result satisfies :math:`x_2^\top F x_1 = 0` for ``(P1, P2)``, the order of :func:`find_fundamental`.- It is not normalised, and for ``P1 = [I | 0]``, ``P2 = [R | t]`` it is the negative of- :func:`~kornia.geometry.epipolar.essential_from_Rt` for the same motion.+ - It is not normalised, and for ``P1 = [I | 0]``, ``P2 = [R | t]`` it is the negative of+ :func:`essential_from_Rt` for the same motion.- float16 and bfloat16 inputs are computed in float32. A float16 ``F`` is then divided by its largestabsolute entry before the cast back, because pixel-unit projection matrices give entries of order ``1e10``,above float16's maximum of 65504; an all-zero ``F`` (coincident camera centres) stays zero. bfloat16 has@@ -804,6 +879,12 @@ def fundamental_from_projections(P1: torch.Tensor, P2: torch.Tensor) -> torch.TeReturns:The fundamental matrix with shape ``(*, 3, 3)``.++ See Also:+ - :func:`projections_from_fundamental`: a canonical camera pair of a fundamental matrix, the reverse up to+ a projective transformation.+ - :func:`essential_from_Rt`: the calibrated counterpart, from two camera poses.+ - :func:`fundamental_from_essential`: a fundamental matrix from an essential matrix and both intrinsics."""KORNIA_CHECK_SHAPE(P1, ["*", "3", "4"])KORNIA_CHECK_SHAPE(P2, ["*", "3", "4"])diff --git a/kornia/geometry/epipolar/numeric.py b/kornia/geometry/epipolar/numeric.pyindex fa652b6b8..bea50c9e3 100644--- a/kornia/geometry/epipolar/numeric.py+++ b/kornia/geometry/epipolar/numeric.py@@ -15,13 +15,18 @@# limitations under the License.#-"""Module containing numerical functionalities for SfM."""+"""Numerical helpers of the epipolar functions: the cross-product matrix and the cofactor matrix."""import torchdef cross_product_matrix(x: torch.Tensor) -> torch.Tensor:- r"""Return the skew-symmetric cross-product matrix of a vector.+ r"""Build the skew-symmetric cross-product matrices of 3-vectors with any number of leading dimensions.++ ``S = cross_product_matrix(x)`` takes vectors ``(*, 3)`` and returns the matrices :math:`[x]_\times`,+ ``(*, 3, 3)``, with ``S @ w == cross(x, w)``: the matrix that :func:`essential_from_Rt` multiplies the relative+ rotation by. :func:`~kornia.geometry.conversions.vector_to_skew_symmetric_matrix` builds the same matrix for a+ ``(3,)`` or ``(B, 3)`` vector only.Convention:- Same matrix as :func:`~kornia.geometry.conversions.vector_to_skew_symmetric_matrix`,@@ -33,6 +38,12 @@ def cross_product_matrix(x: torch.Tensor) -> torch.Tensor:Returns:The skew-symmetric matrix with shape ``(*, 3, 3)``.+ See Also:+ - :func:`~kornia.geometry.conversions.vector_to_skew_symmetric_matrix`: the same matrix, for ``(3,)`` or+ ``(B, 3)`` vectors only.+ - :meth:`~kornia.geometry.liegroup.So3.hat`: the same matrix, as the Lie-algebra element of a rotation+ vector.+"""if not x.shape[-1] == 3:raise AssertionError(x.shape)diff --git a/kornia/geometry/epipolar/projection.py b/kornia/geometry/epipolar/projection.pyindex bb50f87f5..5b037f167 100644--- a/kornia/geometry/epipolar/projection.py+++ b/kornia/geometry/epipolar/projection.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Module for image projections."""+"""Camera intrinsics and projection matrices: compose, decompose and build them for two-view geometry."""from typing import Tuple, Union@@ -30,11 +30,16 @@ from .numeric import cross_product_matrixdef intrinsics_like(focal: float, input: torch.Tensor) -> torch.Tensor:- r"""Return a 3x3 intrinsics matrix per batch element of the input.+ r"""Build a pinhole camera matrix per image of a batch from one focal length and the image size.++ ``K = intrinsics_like(focal, input)`` takes a focal length in pixels and a floating-point image batch+ ``(B, C, H, W)`` and returns ``(B, 3, 3)`` intrinsics with ``fx = fy = focal``, zero skew and the principal point+ ``(W / 2, H / 2)``, in the dtype and on the device of the images. That principal point is the half-pixel centre,+ not the integer-pixel centre of kornia's grids (see the known defect below). For random intrinsics, use+ :func:`random_intrinsics`; to resize existing ones with their image, :func:`scale_intrinsics`.Convention:- - ``fx = fy = focal`` and the principal point is ``(W / 2, H / 2)`` of the ``(B, C, H, W)`` input;- dtype and device follow the input. ``focal`` must be positive and the input 4-D.+ - ``focal`` must be positive and the input 4-D.- The input must be floating point, like kornia's images: an integer input raises by design rather thanreturning an integer ``K``.- Known defects: ``(W / 2, H / 2)`` is the half-pixel centre, not the integer-pixel centre@@ -49,6 +54,12 @@ def intrinsics_like(focal: float, input: torch.Tensor) -> torch.Tensor:Returns:The camera matrix with the shape of ``(B, 3, 3)``.+ See Also:+ - :func:`random_intrinsics`: random focal lengths and principal point instead.+ - :func:`scale_intrinsics`: the intrinsics of a resized image.+ - :class:`~kornia.geometry.camera.pinhole.PinholeCamera`: a camera model that holds intrinsics and+ extrinsics together.+"""if len(input.shape) != 4:raise AssertionError(input.shape)@@ -66,12 +77,16 @@ def intrinsics_like(focal: float, input: torch.Tensor) -> torch.Tensor:def random_intrinsics(low: Union[float, torch.Tensor], high: Union[float, torch.Tensor]) -> torch.Tensor:- r"""Generate a random camera matrix based on a given uniform distribution.+ r"""Draw a random pinhole camera matrix with focal lengths and principal point uniform in one range.++ ``K = random_intrinsics(low, high)`` returns a ``(1, 3, 3)`` camera matrix with zero skew whose ``fx``, ``fy``,+ ``cx`` and ``cy`` are four independent draws from :math:`U(low, high)`, for synthetic data such as that of+ :func:`generate_scene`. For deterministic intrinsics from a focal length and an image size, use+ :func:`intrinsics_like`.Convention:- - ``fx``, ``fy``, ``cx`` and ``cy`` are four draws from :math:`U(low, high)` on torch's global generator.- The bounds are scalars; dtype and device follow tensor bounds, and Python floats give the default dtype- and device.+ - The draws come from torch's global generator. The bounds are scalars; dtype and device follow tensor+ bounds, and Python floats give the default dtype and device.Args:low: lower range (inclusive).@@ -80,6 +95,10 @@ def random_intrinsics(low: Union[float, torch.Tensor], high: Union[float, torch.Returns:the random camera matrix with the shape of ``(1, 3, 3)``.+ See Also:+ - :func:`intrinsics_like`: deterministic intrinsics from a focal length and an image size.+ - :func:`generate_scene`: a random scene whose cameras share one such matrix.+"""sampler = torch.distributions.Uniform(low, high)params = sampler.sample((4,))@@ -89,9 +108,14 @@ def random_intrinsics(low: Union[float, torch.Tensor], high: Union[float, torch.def scale_intrinsics(camera_matrix: torch.Tensor, scale_factor: Union[float, torch.Tensor]) -> torch.Tensor:- r"""Scale a camera matrix containing the intrinsics.+ r"""Scale the focal lengths and principal point of camera matrices by an image resize factor.- Applies the scaling factor to the focal length and center of projection.+ ``K_scaled = scale_intrinsics(camera_matrix, scale_factor)`` takes camera matrices ``(B, 3, 3)`` and a factor, a+ float or a ``(B,)`` tensor, and returns a new tensor of the input's shape with ``fx``, ``fy``, ``cx`` and ``cy``+ multiplied by it, for an image resized by that factor. This is the rule of+ :meth:`~kornia.geometry.camera.pinhole.PinholeCamera.scale`, with its known defects (below): the principal point+ does not follow kornia's integer pixel centres, and the skew is left unscaled. For the camera of a whole+ :class:`~kornia.geometry.camera.pinhole.PinholeCamera`, use that method.Convention:- Applies the rule of :meth:`~kornia.geometry.camera.pinhole.PinholeCamera.scale` and returns a new tensor.@@ -109,6 +133,10 @@ def scale_intrinsics(camera_matrix: torch.Tensor, scale_factor: Union[float, torReturns:The scaled camera matrix with the same shape as the input ``(B, 3, 3)``.+ See Also:+ - :meth:`~kornia.geometry.camera.pinhole.PinholeCamera.scale`: the same rule on a camera model.+ - :func:`intrinsics_like`: new intrinsics from a focal length and an image size.+"""K_scale = camera_matrix.clone()K_scale[..., 0, 0] *= scale_factor@@ -119,16 +147,18 @@ def scale_intrinsics(camera_matrix: torch.Tensor, scale_factor: Union[float, tordef projection_from_KRt(K: torch.Tensor, R: torch.Tensor, t: torch.Tensor) -> torch.Tensor:- r"""Get the projection matrix P from K, R and t.+ r"""Compose the 3x4 projection matrix P = K [R | t] of intrinsics and a world-to-camera pose.- This function computes the product :math:`P = K [R|t]`.+ ``P = projection_from_KRt(K, R, t)`` takes intrinsics ``(*, 3, 3)``, world-to-camera rotations ``(*, 3, 3)``+ and translations ``(*, 3, 1)`` and returns the projection matrices :math:`P = K [R|t]`, ``(*, 3, 4)``, that take+ homogeneous world points to homogeneous pixels. For the reverse, use :func:`KRt_from_projection`; for+ ``[R | t]`` without intrinsics, :func:`~kornia.geometry.transform.projection_from_Rt`.Convention:- ``R`` and ``t`` take world points into the camera frame, ``X_cam = R @ X + t``, and ``K`` takes thecamera frame to pixels. ``t`` is the extrinsic translation; the camera centre is ``-R.T @ t``.- ``K``, ``R`` and ``t`` must have the same number of dimensions, and ``R`` and ``t`` the same batch shape;- ``K`` broadcasts against them. The inverse is- :func:`KRt_from_projection`, and :func:`depth_from_point` gives the depth of a point in this camera.+ ``K`` broadcasts against them. :func:`depth_from_point` gives the depth of a point in this camera.Args:K: the camera matrix with the intrinsics with shape ``(*, 3, 3)``.@@ -138,6 +168,12 @@ def projection_from_KRt(K: torch.Tensor, R: torch.Tensor, t: torch.Tensor) -> toReturns:The projection matrix P with shape ``(*, 3, 4)``.+ See Also:+ - :func:`KRt_from_projection`: the reverse, ``K``, ``R`` and ``t`` of a projection matrix.+ - :func:`~kornia.geometry.transform.projection_from_Rt`: ``[R | t]`` alone, without intrinsics.+ - :func:`depth_from_point`: the depth of points in the camera of this matrix.+ - :func:`triangulate_points`: 3D points from two such matrices and matched pixels.+"""KORNIA_CHECK_SHAPE(K, ["*", "3", "3"])KORNIA_CHECK_SHAPE(R, ["*", "3", "3"])@@ -156,12 +192,17 @@ def projection_from_KRt(K: torch.Tensor, R: torch.Tensor, t: torch.Tensor) -> todef KRt_from_projection(P: torch.Tensor, eps: float = 1e-6) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor]:- r"""Decompose the Projection matrix into ``K``, ``R`` and ``t`` with :math:`K [R|t] = \pm P`.+ r"""Decompose a 3x4 projection matrix into intrinsics, rotation and translation, with K [R | t] = +-P.++ ``K, R, t = KRt_from_projection(P)`` takes projection matrices ``(B, 3, 4)`` and returns, by an RQ decomposition+ of their left ``3 x 3`` block, the intrinsics ``K`` ``(B, 3, 3)``, upper triangular with a positive diagonal, the+ world-to-camera rotation ``R`` ``(B, 3, 3)`` with ``det R = 1`` and the translation ``t`` ``(B, 3, 1)``, not the+ camera centre: the inverse of :func:`projection_from_KRt` up to the scale and sign of ``P``. ``K`` keeps the scale+ of ``P``; divide it by ``K[..., 2:, 2:]`` for ``K[2, 2] = 1``.Convention:- - Inverse of :func:`projection_from_KRt`: returns ``K`` (upper triangular, positive diagonal), a- rotation ``R`` (``det R = 1``) and the translation ``t``, not the camera centre;- :ref:`Two-view geometry <two-view-conventions>` maps this onto OpenCV.+ - :ref:`Two-view geometry <two-view-conventions>` compares this with OpenCV's+ ``decomposeProjectionMatrix``, which returns the camera centre.- ``P`` is defined up to a nonzero scale, sign included: ``s * P`` gives the same ``R`` and ``t`` forevery nonzero ``s`` and ``|s| * K``. So :math:`K [R|t]` is ``P`` when ``P[:, :3, :3]`` has positivedeterminant and ``-P`` when it has negative determinant (``-P`` gives the factors of ``P``). ``K`` is@@ -179,6 +220,11 @@ def KRt_from_projection(P: torch.Tensor, eps: float = 1e-6) -> Tuple[torch.Tenso- The Rotation matrix with shape ``(B, 3, 3)``.- The Translation vector with shape ``(B, 3, 1)``.+ See Also:+ - :func:`projection_from_KRt`: the reverse, ``P = K [R | t]``.+ - :func:`projections_from_fundamental`: a projective camera pair consistent with a fundamental matrix; the+ left ``3 x 3`` block of its second camera has rank 2, so this decomposition gives no metric camera for it.+"""KORNIA_CHECK_SHAPE(P, ["*", "3", "4"])# A caller's autocast region would run the products below in half precision, which torch.linalg.det (CPU) and@@ -221,12 +267,16 @@ def _KRt_from_projection(P: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor, tdef depth_from_point(R: torch.Tensor, t: torch.Tensor, X: torch.Tensor) -> torch.Tensor:- r"""Return the depth of a point transformed by a rigid transform.+ r"""Compute the camera-frame depth of 3D points under a world-to-camera rotation and translation.++ ``depth = depth_from_point(R, t, X)`` takes a world-to-camera rotation ``(*, 3, 3)`` and translation+ ``(*, 3, 1)`` and points ``(*, N, 3)`` and returns the ``z`` coordinate of ``R @ X + t``, ``(*, N)``: the depth+ of each point in the camera of :func:`projection_from_KRt` with the same ``R`` and ``t``, positive in front of+ it. :func:`motion_from_essential_choose_solution` tests its candidate poses with it.Convention:- - Returns the ``z`` coordinate of ``R @ X + t``, the depth in the camera of :func:`projection_from_KRt`- with the same ``R`` and ``t``. The sign is not checked. An ``X`` of shape ``(B, 3)`` with a batched- ``R`` is read as ``B`` points seen by every camera and gives ``(B, B)``, not one point per camera.+ - The sign is not checked. An ``X`` of shape ``(B, 3)`` with a batched ``R`` is read as ``B`` points seen+ by every camera and gives ``(B, B)``, not one point per camera.Args:R: The rotation matrix with shape ``(*, 3, 3)``.@@ -236,6 +286,10 @@ def depth_from_point(R: torch.Tensor, t: torch.Tensor, X: torch.Tensor) -> torchReturns:The depth value per point with shape ``(*, N)``.+ See Also:+ - :func:`projection_from_KRt`: the projection matrix of the camera this depth is measured in.+ - :func:`triangulate_points`: the 3D points of matched pixels, whose depths this measures.+"""X_tmp = R @ X.transpose(-2, -1)return X_tmp[..., 2, :] + t[..., 2, :]@@ -254,12 +308,17 @@ def _nullspace(A: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:def projections_from_fundamental(F_mat: torch.Tensor) -> torch.Tensor:- r"""Get the projection matrices from the Fundamental Matrix.+ r"""Build a canonical pair of projection matrices consistent with a fundamental matrix.++ ``P = projections_from_fundamental(F_mat)`` takes fundamental matrices ``(B, 3, 3)`` in the+ :math:`x_2^\top F x_1 = 0` order of :func:`find_fundamental` and returns the canonical camera pair of Hartley+ and Zisserman, ``(B, 3, 4, 2)`` with the two matrices stacked on the last dimension, for which+ :func:`fundamental_from_projections` gives back ``F`` up to scale. The pair is determined only up to a+ projective transformation of the world, so it is not a metric pose; for the rotation and translation of+ calibrated cameras, use :func:`motion_from_essential_choose_solution` on an essential matrix.Convention:- - Returns the canonical pair for ``F`` in the ``x2^T F x1 = 0`` convention of- :func:`~kornia.geometry.epipolar.find_fundamental`, stacked on the last dimension:- ``[..., 0] = [I | 0]`` for the first image and ``[..., 1] = [[e2]_x F | e2]``, with ``e2`` the+ - ``[..., 0] = [I | 0]`` for the first image and ``[..., 1] = [[e2]_x F | e2]``, with ``e2`` theleft null vector of ``F``, the epipole in the second image (``e2^T F = 0``).- ``F_mat`` must have exactly one batch dimension.@@ -269,6 +328,11 @@ def projections_from_fundamental(F_mat: torch.Tensor) -> torch.Tensor:Returns:The projection matrices with shape ``(B, 3, 4, 2)``.+ See Also:+ - :func:`fundamental_from_projections`: the reverse, the fundamental matrix of two projection matrices.+ - :func:`triangulate_points`: a projective reconstruction of matched pixels from this pair.+ - :func:`KRt_from_projection`: the decomposition of a metric projection matrix.+"""KORNIA_CHECK_SHAPE(F_mat, ["*", "3", "3"])diff --git a/kornia/geometry/epipolar/scene.py b/kornia/geometry/epipolar/scene.pyindex 8a99a4cd4..138df7023 100644--- a/kornia/geometry/epipolar/scene.py+++ b/kornia/geometry/epipolar/scene.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Generate synthetic 3d scenes."""+"""Generate random synthetic scenes to test the two-view and multi-view estimators on."""import mathfrom typing import Dict@@ -31,6 +31,11 @@ from .projection import projection_from_KRt, random_intrinsicsdef generate_scene(num_views: int, num_points: int) -> Dict[str, torch.Tensor]:"""Generate a random 3d scene seen by ``num_views`` cameras that share one camera matrix.+ ``scene = generate_scene(num_views, num_points)`` returns a dictionary with a random camera matrix from+ :func:`random_intrinsics`, the world-to-camera rotation, translation and projection matrix of each view,+ ``num_points`` random 3D points in front of every view and their pixel projections, ``points2d``. Pass two of its+ views to :func:`find_fundamental` or :func:`triangulate_points` to test them against the known geometry.+Convention:- Every point has positive depth in every view; depth and focal length are not bounded away from zero.- Draws from torch's global generator.@@ -49,6 +54,10 @@ def generate_scene(num_views: int, num_points: int) -> Dict[str, torch.Tensor]:- ``"points3d"``: the 3d points, in :math:`[0, 1)` per coordinate, with shape ``(1, N, 3)``.- ``"points2d"``: the projection of ``points3d`` by each ``P``, with shape ``(V, N, 2)``.+ See Also:+ - :func:`random_intrinsics`: the camera matrix of the scene.+ - :func:`projection_from_KRt`: the projection matrix of each view.+"""# Generate the 3d pointspoints3d = torch.rand(1, num_points, 3) # NxMx3diff --git a/kornia/geometry/epipolar/triangulation.py b/kornia/geometry/epipolar/triangulation.pyindex 8e00f794d..ff166eaf3 100644--- a/kornia/geometry/epipolar/triangulation.py+++ b/kornia/geometry/epipolar/triangulation.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Module with the functionalities for triangulation."""+"""Triangulate 3D points from two views by the DLT."""from __future__ import annotations@@ -105,13 +105,16 @@ def triangulate_points(points2: torch.Tensor,solver: str = "eigh",) -> torch.Tensor:- r"""Reconstructs a bunch of points by triangulation.+ r"""Triangulate 3D points from their projections in two views by the linear DLT.- Triangulates the 3d position of 2d correspondences between two images.- Reference: the ``"svd"`` and ``"eigh"`` solvers use the DLT formulation from Hartley/Zisserman 12.2 pag.312.-- The input points are assumed to be inlier correspondences. The method does not perform any robust- estimation.+ ``points3d = triangulate_points(P1, P2, points1, points2)`` takes the projection matrices ``(*, 3, 4)`` of two+ cameras and matched points ``(*, N, 2)`` in their image coordinates and returns the Euclidean 3D points+ ``(*, N, 3)`` in the frame that ``P1`` and ``P2`` map from: the world frame for ``P = K [R | t]`` from+ :func:`projection_from_KRt`, and the frame of camera 1 for ``P1 = K1 [I | 0]``. Each point is the least-squares+ null vector of its four DLT equations (Hartley and Zisserman, 12.2, p. 312) with ``solver="svd"`` or ``"eigh"``,+ and its approximation by one step of inverse iteration with ``"cofactor"``. The correspondences are assumed to be+ inliers: there is no robust estimation and no cheirality check. For the relative pose of two calibrated cameras+ together with the points, use :func:`motion_from_essential_choose_solution`.Convention:- ``P1`` pairs with ``points1`` and ``P2`` with ``points2``. Input and output points are Euclidean;@@ -162,6 +165,12 @@ def triangulate_points(Returns:The reconstructed 3d points in the world frame with shape ``(*, N, 3)``.+ See Also:+ - :func:`motion_from_essential_choose_solution`: the relative pose from an essential matrix, with the points+ triangulated in the frame of camera 1.+ - :func:`projection_from_KRt`: the projection matrices to pass, from intrinsics and poses.+ - :func:`~kornia.geometry.solvers.null_vector_3x4`: the cofactor null vector that ``"cofactor"`` builds on.+Example:>>> P1 = torch.eye(3, 4)[None] # 1x3x4>>> P2 = torch.eye(3, 4)[None]diff --git a/kornia/geometry/homography.py b/kornia/geometry/homography.pyindex bce7b4ded..205fb8e8b 100644--- a/kornia/geometry/homography.py+++ b/kornia/geometry/homography.py@@ -15,6 +15,35 @@# limitations under the License.#+r"""Estimate homographies from point or line-segment correspondences, and score them.++A homography ``H`` ``(B, 3, 3)`` maps the first image to the second, :math:`x_2 \sim H x_1` for homogeneous+points, as ``cv2.findHomography(src, dst)`` does: the source-to-destination pixel homography that+:func:`~kornia.geometry.transform.warp_perspective` takes. Correspondences are ``(B, N, 2)`` points, or+``(B, N, 2, 2)`` segments as ``[start, end]`` pairs of ``(x, y)`` points, with the first image's argument first. The+estimators scale ``H`` to ``H[2, 2] = 1``, unless that entry is numerically zero. The errors do not depend on the+scale of ``H`` and are distances in the units of the points: squared by default for points, unsquared for segments.+:ref:`Two-view geometry <two-view-conventions>` compares these conventions with OpenCV's.++Which function to use:++- A homography from correspondences known to be inliers: :func:`find_homography_dlt` from four or more points,+ optionally weighted, and :func:`find_homography_lines_dlt` from four or more line segments.+ :func:`find_homography_dlt_iterated` and :func:`find_homography_lines_dlt_iterated` re-weight them by their+ residuals over a few iterations. Exactly four points in closed form:+ :func:`~kornia.geometry.transform.get_perspective_transform`. With outliers:+ :class:`~kornia.geometry.ransac.RANSAC`.+- How well correspondences fit a homography: :func:`oneway_transfer_error` measures in image 2 only,+ :func:`symmetric_transfer_error` in both images through ``H`` and its inverse, and+ :func:`sampson_homography_distance` is the first-order error of moving both points.+ :func:`line_segment_transfer_error_one_way` scores segments by the distance of their mapped endpoints from the+ matched line.+- Whether four correspondences keep their orientation, as two views of a plane in front of both cameras do, before+ fitting them: :func:`sample_is_valid_for_homography`. A mirroring homography fails this test.+- A two-view model other than a plane: :func:`~kornia.geometry.epipolar.find_fundamental`,+ :func:`~kornia.geometry.epipolar.find_essential`.+"""+import mathimport warningsfrom typing import Optional, Tuple, Union@@ -54,11 +83,17 @@ TupleTensor = Tuple[torch.Tensor, torch.Tensor]def oneway_transfer_error(pts1: torch.Tensor, pts2: torch.Tensor, H: torch.Tensor, squared: bool = True, eps: float = 1e-8) -> torch.Tensor:- r"""Return transfer error in image 2 for correspondences given the homography matrix.+ r"""Compute the squared transfer error in image 2 of correspondences under a homography.++ ``err = oneway_transfer_error(pts1, pts2, H)`` takes correspondences ``(B, N, 2)``, or homogeneous+ ``(B, N, 3)``, and homographies ``(B, 3, 3)`` that map the first image to the second, and returns, per+ correspondence, ``(B, N)`` with the batch dimensions broadcast, the squared distance in image 2 between ``H``+ applied to ``pts1`` and ``pts2``, in the squared units of the points; ``squared=False`` returns the distance. It+ is the residual :class:`~kornia.geometry.ransac.RANSAC` scores homographies with. It measures in image 2 only:+ for both images, use :func:`symmetric_transfer_error`, and for the first-order error of moving both points,+ :func:`sampson_homography_distance`.Convention:- - ``oneway_transfer_error(pts1, pts2, H)`` measures in image 2, between ``H`` applied to ``pts1`` and- ``pts2``.- One set of correspondences scored against several homographies (points with leading dimensions ofsize 1) is computed from one matrix product for all of them, in at least float32; the result matches theper-homography computation to roundoff.@@ -69,9 +104,9 @@ def oneway_transfer_error(Args:pts1: correspondences from the left images with shape- (B, N, 2 or 3). If they are homogeneous, converted automatically.+ ``(B, N, 2)`` or ``(B, N, 3)``. If they are homogeneous, converted automatically.pts2: correspondences from the right images with shape- (B, N, 2 or 3). If they are homogeneous, converted automatically.+ ``(B, N, 2)`` or ``(B, N, 3)``. If they are homogeneous, converted automatically.H: Homographies with shape ``(B, 3, 3)``.squared: if True (default), the squared distance is returned.eps: unused; it was added to the projective denominator and inside the square root, which made the error@@ -81,6 +116,13 @@ def oneway_transfer_error(Returns:the computed distance with shape ``(B, N)``.+ See Also:+ - :func:`symmetric_transfer_error`: the image-1 error of the inverse homography added.+ - :func:`sampson_homography_distance`: the first-order error of moving both points.+ - :func:`line_segment_transfer_error_one_way`: the image-2 error of line segments.+ - :func:`~kornia.geometry.epipolar.left_to_right_epipolar_distance`: the image-2 error of a fundamental+ matrix.+"""KORNIA_CHECK_SHAPE(H, ["B", "3", "3"])if H.shape[0] >= 2 and _shares_points(pts1, pts2):@@ -134,19 +176,24 @@ def oneway_transfer_error(def symmetric_transfer_error(pts1: torch.Tensor, pts2: torch.Tensor, H: torch.Tensor, squared: bool = True, eps: float = 1e-8) -> torch.Tensor:- r"""Return Symmetric transfer error for correspondences given the homography matrix.+ r"""Compute the symmetric transfer error of correspondences under a homography, in both images.++ ``err = symmetric_transfer_error(pts1, pts2, H)`` takes the arguments of :func:`oneway_transfer_error` and+ returns, per correspondence, ``(B, N)``, the squared image-2 transfer error of ``H`` plus the squared image-1+ transfer error of :math:`H^{-1}`; ``squared=False`` returns the square root of that sum. A homography that is not+ invertible scores the largest finite value of the dtype. :func:`find_homography_dlt_iterated` re-weights its+ correspondences by it. For the first-order error of moving both points, use :func:`sampson_homography_distance`.Convention:- - Argument order as :func:`oneway_transfer_error`. The squared value is the image-2 error of ``H`` plus- the image-1 error of ``H^-1``, and ``squared=False`` returns the square root of that sum.+ - Argument order as :func:`oneway_transfer_error`.- As :func:`oneway_transfer_error`, the error does not depend on the scale of ``H`` and an exact matchscores exactly 0 with ``squared=False``.Args:pts1: correspondences from the left images with shape- (B, N, 2 or 3). If they are homogeneous, converted automatically.+ ``(B, N, 2)`` or ``(B, N, 3)``. If they are homogeneous, converted automatically.pts2: correspondences from the right images with shape- (B, N, 2 or 3). If they are homogeneous, converted automatically.+ ``(B, N, 2)`` or ``(B, N, 3)``. If they are homogeneous, converted automatically.H: Homographies with shape ``(B, 3, 3)``.squared: if True (default), the squared distance is returned.eps: unused, as in :func:`oneway_transfer_error` (#4881).@@ -155,6 +202,12 @@ def symmetric_transfer_error(the computed distance with shape ``(B, N)``. Rows whose homography is not invertiblescore ``max_num``, i.e. ``torch.finfo(pts1.dtype).max``, for both values of ``squared``.+ See Also:+ - :func:`oneway_transfer_error`: the image-2 term alone.+ - :func:`sampson_homography_distance`: the first-order error of moving both points, without an inverse.+ - :func:`~kornia.geometry.epipolar.symmetrical_epipolar_distance`: the two-image error of a fundamental+ matrix.+"""KORNIA_CHECK_SHAPE(H, ["B", "3", "3"])if pts1.size(-1) == 3:@@ -189,7 +242,14 @@ def symmetric_transfer_error(def sampson_homography_distance(pts1: torch.Tensor, pts2: torch.Tensor, H: torch.Tensor, squared: bool = True) -> torch.Tensor:- r"""Return the Sampson distance of correspondences to homographies.+ r"""Compute the Sampson distance of correspondences to a homography, to first order.++ ``d = sampson_homography_distance(pts1, pts2, H)`` takes correspondences ``(B, N, 2)``, or homogeneous+ ``(B, N, 3)``, and homographies ``(B, 3, 3)`` that map ``pts1`` to ``pts2``, and returns, per correspondence,+ ``(B, N)``, the squared Sampson distance; ``squared=False`` returns its square root. Unlike+ :func:`oneway_transfer_error` and :func:`symmetric_transfer_error`, it accounts for the noise of both points in+ one term and needs no inverse homography. For a fundamental matrix, use+ :func:`~kornia.geometry.epipolar.sampson_epipolar_distance`.The first-order approximation of the geometric error: the squared distance by which ``pts1`` and ``pts2``together must move for :math:`x_2 \sim H x_1` to hold (Hartley and Zisserman, *Multiple View Geometry*, 2nd ed.,@@ -218,6 +278,12 @@ def sampson_homography_distance(Returns:the computed distance with shape ``(B, N)``.+ See Also:+ - :func:`oneway_transfer_error`: the error in image 2 only, with all the noise on the second point.+ - :func:`symmetric_transfer_error`: the errors of ``H`` in image 2 and of its inverse in image 1, added.+ - :func:`~kornia.geometry.epipolar.sampson_epipolar_distance`: the same first-order error for a fundamental+ matrix.+"""KORNIA_CHECK_SHAPE(H, ["B", "3", "3"])KORNIA_CHECK(pts1.shape[-1] in (2, 3) and pts2.shape[-1] in (2, 3), "points must have 2 or 3 coordinates")@@ -267,10 +333,15 @@ def sampson_homography_distance(def line_segment_transfer_error_one_way(ls1: torch.Tensor, ls2: torch.Tensor, H: torch.Tensor, squared: bool = False) -> torch.Tensor:- r"""Return the mean perpendicular transfer distance in image 2 for line segment correspondences.+ r"""Compute the mean distance of image-1 segment endpoints, mapped to image 2, from their lines.- Both endpoints of each image-1 segment are mapped into image 2 by ``H`` and scored against the line through- the matching image-2 segment. See :cite:`homolines2001` for details.+ ``err = line_segment_transfer_error_one_way(ls1, ls2, H)`` takes segment correspondences ``(B, N, 2, 2)`` and+ homographies ``(B, 3, 3)`` that map the first image to the second, and returns, per correspondence, ``(B, N)``,+ the mean perpendicular distance in image 2 of the two endpoints of the image-1 segment, mapped by ``H``, from the+ line through the image-2 segment (:cite:`homolines2001`). Unlike the point errors, it is unsquared by default;+ ``squared=True`` returns its square. A zero-length image-2 segment defines no line and scores 0 up to rounding+ (up to 0.015 px in float32 for coordinates of a few hundred pixels). For point correspondences, use+ :func:`oneway_transfer_error`.Convention:- Argument order and direction as :func:`oneway_transfer_error`.@@ -279,15 +350,19 @@ def line_segment_transfer_error_one_way(Args:ls1: line segment correspondences from the left images with shape- (B, N, 2, 2).+ ``(B, N, 2, 2)``.ls2: line segment correspondences from the right images with shape- (B, N, 2, 2).+ ``(B, N, 2, 2)``.H: Homographies with shape ``(B, 3, 3)``.squared: if True (default is False), the squared error is returned.Returns:the computed error with shape ``(B, N)``.+ See Also:+ - :func:`oneway_transfer_error`: the image-2 error of point correspondences.+ - :func:`find_homography_lines_dlt`: the homography that fits line segment correspondences.+"""KORNIA_CHECK_SHAPE(H, ["B", "3", "3"])KORNIA_CHECK_SHAPE(ls1, ["B", "N", "2", "2"])@@ -502,15 +577,21 @@ def _refine_homography_lm(def find_homography_dlt(points1: torch.Tensor, points2: torch.Tensor, weights: Optional[torch.Tensor] = None, solver: str = "lu") -> torch.Tensor:- r"""Compute the homography matrix using the DLT formulation.+ r"""Estimate a homography from four or more weighted point correspondences by the normalized DLT.- The weighted DLT system of four or more correspondences is solved with ``solver``.+ ``H = find_homography_dlt(points1, points2)`` takes correspondences ``(B, N, 2)``, ``N >= 4``, and returns the+ homography ``(B, 3, 3)`` that maps ``points1`` to ``points2``, :math:`x_2 \sim H x_1`: the least-squares+ solution of the Hartley-normalized DLT system, solved with ``solver``, and the source-to-destination pixel+ homography that :func:`~kornia.geometry.transform.warp_perspective` takes. ``weights`` ``(B, N)`` weight each+ correspondence's equations. It is not robust to wrong correspondences: with outliers, use+ :class:`~kornia.geometry.ransac.RANSAC`. For exactly four points in closed form, use+ :func:`~kornia.geometry.transform.get_perspective_transform`; for line segments,+ :func:`find_homography_lines_dlt`.Convention:- - ``H`` maps ``points1`` to ``points2``, ``points2 ~ H @ points1``, and is scaled so that- ``H[2, 2] = 1`` by :func:`~kornia.geometry.epipolar.normalize_transformation`, which leaves it at its- unnormalised scale when ``|H[2, 2]|`` is at most ``1e-8``; :ref:`two-view-conventions` compares this- with OpenCV.+ - ``H`` is scaled so that ``H[2, 2] = 1`` by :func:`~kornia.geometry.epipolar.normalize_transformation`,+ which leaves it at its unnormalised scale when ``|H[2, 2]|`` is at most ``1e-8``;+ :ref:`two-view-conventions` compares this with OpenCV.- ``weights`` multiply each correspondence's squared algebraic residual: a weight of 0 removes thecorrespondence from the equations, and only relative weights matter.- ``solver="lu"`` and ``"svd"`` give the same homography on exact data, to roundoff scaled by the@@ -532,6 +613,19 @@ def find_homography_dlt(``solver="svd"`` fails to converge on a finite system, the whole batch is NaN and a ``RuntimeWarning`` isissued.+ See Also:+ - :func:`find_homography_dlt_iterated`: the same fit re-weighted by the transfer error over a few solves.+ - :func:`find_homography_lines_dlt`: a homography from line segment correspondences.+ - :func:`~kornia.geometry.transform.get_perspective_transform`: the exact homography of four points, in+ closed form.+ - :class:`~kornia.geometry.ransac.RANSAC`: a homography robust to wrong correspondences.+ - :func:`~kornia.geometry.epipolar.find_fundamental`: the two-view model of a general scene instead of a+ plane.+ - :func:`~kornia.geometry.epipolar.find_essential`: the two-view model of calibrated cameras in a general+ scene.+ - :func:`~kornia.geometry.solvers.find_affine`: an affine map, the special case with bottom row+ ``[0, 0, 1]``.+"""device, dtype = _extract_device_dtype([points1, points2])# Weights take the points' dtype, as a caller's autocast used to cast them; mixed dtypes now fit outside it too.@@ -721,7 +815,16 @@ def _homography_from_dlt_system(def find_homography_dlt_iterated(points1: torch.Tensor, points2: torch.Tensor, weights: torch.Tensor, soft_inl_th: float = 3.0, n_iter: int = 5) -> torch.Tensor:- r"""Compute the homography matrix using the iteratively-reweighted least squares (IRWLS).+ r"""Estimate a homography by iteratively re-weighted DLT, with Gaussian weights of the transfer error.++ ``H = find_homography_dlt_iterated(points1, points2, weights)`` takes correspondences ``(B, N, 2)`` and initial+ weights ``(B, N)`` and returns the homography ``(B, 3, 3)`` that maps ``points1`` to ``points2``, as+ :func:`find_homography_dlt` does, after ``n_iter`` weighted DLT solves: each solve after the first re-weights the+ correspondences by a Gaussian of their symmetric transfer error under the previous estimate, of standard+ deviation ``soft_inl_th`` pixels. A correspondence whose symmetric error is ``e = 3 * soft_inl_th`` keeps a+ weight of ``exp(-4.5)``, about 0.01, so the iterations suppress a few outliers when the first solve is close; for+ many outliers use :class:`~kornia.geometry.ransac.RANSAC`, which refines its homography with this function under+ ``local_optimization="dlt"``.Convention:- Direction, ``H[2, 2] = 1`` and autocast as :func:`find_homography_dlt`. Each solve after the first re-weights@@ -741,6 +844,11 @@ def find_homography_dlt_iterated(the computed homography matrix with shape ``(B, 3, 3)``, NaN for a batch element as in:func:`find_homography_dlt`.+ See Also:+ - :func:`find_homography_dlt`: a single weighted solve.+ - :func:`find_homography_lines_dlt_iterated`: the same re-weighting for line segment correspondences.+ - :func:`symmetric_transfer_error`: the error the weights are computed from.+"""device, dtype = _extract_device_dtype([points1, points2])# Weights take the points' dtype, as a caller's autocast used to cast them; mixed dtypes now fit outside it too.@@ -762,13 +870,18 @@ def find_homography_dlt_iterated(def sample_is_valid_for_homography(points1: torch.Tensor, points2: torch.Tensor) -> torch.Tensor:- """Implement oriented constraint check from :cite:`Marquez-Neila2015`.+ """Test whether four correspondences keep their orientation across two views.- Analogous to https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/usac/degeneracy.cpp#L88+ ``valid = sample_is_valid_for_homography(points1, points2)`` takes minimal samples ``(B, 4, 2)`` of the first and+ the second image, or ``(B, N, 2)`` of which the first four points are read, and returns a ``(B,)`` bool mask that+ is True where each of the four triples of points has the same orientation in both images: the oriented+ constraint of :cite:`Marquez-Neila2015`, analogous to OpenCV's USAC degeneracy test+ (https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/usac/degeneracy.cpp#L88).+ :class:`~kornia.geometry.ransac.RANSAC` uses it to discard minimal samples for ``"homography"`` before fitting+ them.Convention:- - :class:`~kornia.geometry.ransac.RANSAC` uses it to discard minimal samples for ``"homography"``. It- checks only that the four triples of the first four points keep their orientation across the two+ - It checks only that the four triples of the first four points keep their orientation across the twoviews, so a mirror-image sample is rejected. A triple that is collinear in both views, for instancethrough a repeated point, counts as kept, so collinearity alone does not reject a sample.@@ -779,7 +892,11 @@ def sample_is_valid_for_homography(points1: torch.Tensor, points2: torch.Tensor)points are used.Returns:- Mask with the minimal sample is good for homography estimation ``(B)``.+ Mask with the minimal sample is good for homography estimation ``(B,)``.++ See Also:+ - :func:`find_homography_dlt`: the fit of a sample that passes.+ - :class:`~kornia.geometry.ransac.RANSAC`: the estimator that draws the samples this tests."""if points1.shape != points2.shape:@@ -811,15 +928,19 @@ def sample_is_valid_for_homography(points1: torch.Tensor, points2: torch.Tensor)def find_homography_lines_dlt(ls1: torch.Tensor, ls2: torch.Tensor, weights: Optional[torch.Tensor] = None) -> torch.Tensor:- """Compute the homography matrix from line segment correspondences with a DLT formulation.+ """Estimate a homography from four or more line segment correspondences by the DLT.- See :cite:`homolines2001` for details.+ ``H = find_homography_lines_dlt(ls1, ls2)`` takes segment correspondences ``(B, N, 2, 2)``, ``N >= 4``, each a+ ``[start, end]`` pair of ``(x, y)`` points, and returns the homography ``(B, 3, 3)`` that maps image-1 points to+ image-2 points, scaled to ``H[2, 2] = 1`` unless that entry is numerically zero, from the constraint that both+ endpoints of each image-1 segment map onto the line through the matching image-2 segment (:cite:`homolines2001`).+ The endpoints need not be point correspondences, so segments detected with different extents still constrain+ ``H``. For point correspondences, use :func:`find_homography_dlt`; with outliers,+ :class:`~kornia.geometry.ransac.RANSAC` with ``model_type="homography_from_linesegments"``.Convention:- - ``H`` maps image-1 points to image-2 points, as in :func:`find_homography_dlt`. Each segment is a- ``[start, end]`` pair of ``(x, y)`` points, and ``weights`` has one entry per segment.- - Both endpoints of image-1 segment ``i`` are constrained to lie, after mapping by ``H``, on the line- through image-2 segment ``i``, so the endpoints need not be point correspondences.+ - ``H`` maps image-1 points to image-2 points, as in :func:`find_homography_dlt`. ``weights`` has one entry+ per segment.- Each segment gives two constraints on the eight degrees of freedom of ``H``, so at least four segmentsare needed, as :func:`find_homography_dlt` needs four points.- A caller's autocast region does not change the fit.@@ -841,6 +962,14 @@ def find_homography_lines_dlt(Raises:AssertionError: if fewer than four segments are given, as in :func:`find_homography_dlt`.+ See Also:+ - :func:`find_homography_dlt`: a homography from point correspondences.+ - :func:`find_homography_lines_dlt_iterated`: the same fit re-weighted by the line distance over a few+ solves.+ - :func:`line_segment_transfer_error_one_way`: the residual of segments under the result.+ - :class:`~kornia.geometry.ransac.RANSAC`: a homography from segments with outliers, with+ ``model_type="homography_from_linesegments"``.+"""if len(ls1.shape) == 3:ls1 = ls1[None]@@ -912,7 +1041,14 @@ def find_homography_lines_dlt(def find_homography_lines_dlt_iterated(ls1: torch.Tensor, ls2: torch.Tensor, weights: torch.Tensor, soft_inl_th: float = 4.0, n_iter: int = 5) -> torch.Tensor:- r"""Compute the homography matrix using the iteratively-reweighted least squares (IRWLS) from line segments.+ r"""Estimate a homography from line segments by iteratively re-weighted DLT.++ ``H = find_homography_lines_dlt_iterated(ls1, ls2, weights)`` takes segment correspondences ``(B, N, 2, 2)``,+ ``N >= 4``, and initial weights ``(B, N)`` and returns the homography ``(B, 3, 3)`` that maps image-1 points to+ image-2 points, as :func:`find_homography_lines_dlt` does, after ``n_iter`` weighted solves: each solve after the+ first re-weights the segments by a Gaussian, of standard deviation ``soft_inl_th`` pixels, of the mean distance+ of their mapped endpoints from the matched lines under the previous estimate. For point correspondences, use+ :func:`find_homography_dlt_iterated`.Convention:- As :func:`find_homography_dlt_iterated`, with :func:`find_homography_lines_dlt` as the solver and, as@@ -939,6 +1075,11 @@ def find_homography_lines_dlt_iterated(Raises:AssertionError: if fewer than four segments are given, as in :func:`find_homography_lines_dlt`.+ See Also:+ - :func:`find_homography_lines_dlt`: a single weighted solve.+ - :func:`find_homography_dlt_iterated`: the same re-weighting for point correspondences.+ - :func:`line_segment_transfer_error_one_way`: the distance the weights are computed from.+"""# Promote unbatched segments as find_homography_lines_dlt does: the re-weighting below needs the batch axis.if len(ls1.shape) == 3:diff --git a/kornia/geometry/pose.py b/kornia/geometry/pose.pyindex e0157715b..5a599bbaf 100644--- a/kornia/geometry/pose.py+++ b/kornia/geometry/pose.py@@ -15,6 +15,18 @@# limitations under the License.#+"""Rigid SE(2) and SE(3) poses that carry the names of their source and destination frames.++A pose ``b_from_a`` maps points in frame ``a`` to frame ``b``, and ``c_from_b * b_from_a`` is ``c_from_a``: the+names make a composition whose frames do not chain raise instead of returning a wrong pose.++Which class to use:++- A pose between named frames, built from an :class:`~kornia.geometry.liegroup.Se3` or+ :class:`~kornia.geometry.liegroup.Se2`, a rotation and a translation, or a matrix: :class:`NamedPose`.+- The pose without names: :class:`~kornia.geometry.liegroup.Se3` and :class:`~kornia.geometry.liegroup.Se2`.+"""+from __future__ import annotationsimport uuid@@ -43,21 +55,34 @@ def _check_matrix_shape(matrix: torch.Tensor, matrix_type: str = "R") -> None:class NamedPose:- r"""Class to represent a named pose between two frames.-- Internally represented by either Se2 or Se3.+ r"""Hold an SE(2) or SE(3) pose with the names of its two frames, which composition checks.++ ``b_from_a = NamedPose(pose, frame_src="a", frame_dst="b")`` wraps an :class:`~kornia.geometry.liegroup.Se3` or+ :class:`~kornia.geometry.liegroup.Se2` pose that maps frame ``a`` to frame ``b``;+ ``b_from_a.transform_points(points)`` maps points from ``a`` to ``b``, and ``c_from_b * b_from_a`` composes two+ poses into ``c_from_a`` after checking that the frames chain. An unbatched ``Se3`` maps one point ``(3,)`` and a+ batch of ``B`` maps ``(B, 3)``, one point per pose (a batch of one also maps ``(N, 3)``); a point set ``(N, 3)``+ with an unbatched ``Se3`` raises (`#6426 <https://github.com/kornia/kornia/issues/6426>`_). An unbatched ``Se2``+ maps ``(2,)`` or ``(N, 2)``, and a batch of ``B`` maps ``(B, 2)``. Build one from a rotation and a translation with+ :meth:`from_rt`, or from a ``3 x 3`` or ``4 x 4`` matrix with :meth:`from_matrix`. For a pose without frame names,+ use the Lie group classes directly; for ``4 x 4`` matrices,+ :func:`~kornia.geometry.linalg.compose_transformations`.Convention:- - ``NamedPose(pose, frame_src="a", frame_dst="b")`` is ``b_from_a``: :meth:`transform_points` maps points in- frame ``a`` to frame ``b``, so ``NamedPose(pose, frame_src="1", frame_dst="0")`` is ``trans_01`` in the- notation of :func:`~kornia.geometry.linalg.relative_transformation`, for an ``Se2`` and an ``Se3`` pose alike.- - ``c_from_b * b_from_a`` is ``c_from_a``, whose matrix is ``c_from_b.pose.matrix() @ b_from_a.pose.matrix()``.+ - ``NamedPose(pose, frame_src="1", frame_dst="0")`` is ``trans_01`` in the notation of+ :func:`~kornia.geometry.linalg.relative_transformation`, for an ``Se2`` and an ``Se3`` pose alike.+ - The matrix of ``c_from_b * b_from_a`` is ``c_from_b.pose.matrix() @ b_from_a.pose.matrix()``.The left operand's ``frame_src`` must equal the right operand's ``frame_dst``, otherwise ``*`` raises``ValueError``; :meth:`inverse` inverts the pose and swaps the frame names.- The pose must be an ``Se2`` or an ``Se3``: an ``So3``, ``So2``, ``Quaternion`` or tensor is rejected atconstruction (use :meth:`from_rt` or :meth:`from_matrix` to build a pose from one), and ``*`` raises``TypeError`` for a right operand that is not a ``NamedPose`` or holds the other group.+ See Also:+ - :class:`~kornia.geometry.liegroup.Se3`: the 3D rigid pose it wraps, without frame names.+ - :class:`~kornia.geometry.liegroup.Se2`: the 2D rigid pose it wraps, without frame names.+ - :func:`~kornia.geometry.linalg.relative_transformation`: the relative pose of two ``4 x 4`` matrices.+Example:>>> b_from_a = NamedPose(Se3.identity(), frame_src="frame_a", frame_dst="frame_b")>>> b_from_adiff --git a/kornia/geometry/ransac.py b/kornia/geometry/ransac.pyindex 379790037..6098ff423 100644--- a/kornia/geometry/ransac.py+++ b/kornia/geometry/ransac.py@@ -15,7 +15,23 @@# limitations under the License.#-"""Module containing RANSAC modules."""+r"""Estimate geometric models robustly from correspondences that contain outliers, with RANSAC.++:class:`RANSAC` fits one model to one set of ``(N, 2)`` correspondences (``(N, 2, 2)`` segments for+``"homography_from_linesegments"``), with the first image's points first: a homography or an affine map from the first+image to the second, a fundamental matrix with :math:`x_2^\top F x_1 = 0` from pixels, or an essential matrix from+normalized camera coordinates. Its threshold ``inl_th`` is a distance in the units of the correspondences.+:ref:`Two-view geometry <two-view-conventions>` compares this with OpenCV.++Which class to use:++- One model from correspondences with outliers: :class:`RANSAC`, with ``model_type`` one of ``"homography"``,+ ``"homography_from_linesegments"``, ``"affine"``, ``"fundamental"`` (seven-point samples),+ ``"fundamental_7pt"``, ``"fundamental_8pt"`` or ``"essential"``.+- A model from correspondences known to be inliers, without sampling: the estimators it wraps,+ :func:`~kornia.geometry.homography.find_homography_dlt`, :func:`~kornia.geometry.solvers.find_affine`,+ :func:`~kornia.geometry.epipolar.find_fundamental` and :func:`~kornia.geometry.epipolar.find_essential`.+"""from __future__ import annotations@@ -178,13 +194,24 @@ def _check_compile_config(local_optimization: str, prosac_sampling: bool, degensclass RANSAC(nn.Module):- """Module for robust geometry estimation with RANSAC. https://en.wikipedia.org/wiki/Random_sample_consensus.+ r"""Estimate a homography, affine, fundamental or essential matrix robustly despite outliers.++ ``estimator = RANSAC(model_type, inl_th=2.0)`` builds the estimator, and ``model, inliers = estimator(kp1, kp2)``+ takes one set of correspondences ``(N, 2)`` of the first and the second image, or segments ``(N, 2, 2)`` for+ ``"homography_from_linesegments"``, and returns the best-scoring ``(3, 3)`` model and its ``(N,)`` bool inlier+ mask, or an all-zero model and an all-False mask when no model has enough support. A correspondence is an inlier+ when its error under the model is at most ``inl_th``, in the units of the keypoints. The model follows the+ estimator of its type: a homography or affine map from ``kp1`` to ``kp2``+ (:func:`~kornia.geometry.homography.find_homography_dlt`, :func:`~kornia.geometry.solvers.find_affine`), a+ fundamental matrix with :math:`x_2^\top F x_1 = 0` (:func:`~kornia.geometry.epipolar.find_fundamental`), or an+ essential matrix for camera-normalized keypoints (:func:`~kornia.geometry.epipolar.find_essential`). Each call+ estimates one model, since ``batch_size`` batches hypotheses rather than problems; for correspondences known to be+ inliers, call those estimators directly (see also https://en.wikipedia.org/wiki/Random_sample_consensus).Convention:- ``kp1`` and ``kp2`` are passed as the first- and second-image arguments of the estimator selected by- ``model_type``, so a homography maps ``kp1`` to ``kp2``. ``forward`` returns a ``(3, 3)`` model and an- ``(N,)`` bool inlier mask, or an all-zero model and no inliers when no candidate has more inliers than- its minimal sample (three correspondences for affine transforms, four for homographies,+ ``model_type``. ``forward`` returns an all-zero model and no inliers when no candidate has more inliers+ than its minimal sample (three correspondences for affine transforms, four for homographies,five for ``"essential"``, seven for ``"fundamental"`` and ``"fundamental_7pt"``,eight for ``"fundamental_8pt"``).- ``"fundamental"`` draws seven-point samples, like ``"fundamental_7pt"``; ``"fundamental_8pt"`` draws@@ -283,6 +310,15 @@ class RANSAC(nn.Module):it off. True enables it for ``"fundamental"`` and ``"fundamental_7pt"`` with ``local_optimization="lm"``,the only supported combinations; True with any other raises ``ValueError``.+ See Also:+ - :func:`~kornia.geometry.homography.find_homography_dlt`: the homography of correspondences known to be+ inliers.+ - :func:`~kornia.geometry.epipolar.find_fundamental`: the fundamental matrix of inlier correspondences.+ - :func:`~kornia.geometry.epipolar.find_essential`: the essential matrix candidates of inlier+ correspondences in normalized camera coordinates.+ - :func:`~kornia.geometry.solvers.find_affine`: the affine map of inlier correspondences.+ - :func:`~kornia.geometry.homography.find_homography_lines_dlt`: the homography of inlier line segments.+"""def __init__(@@ -469,7 +505,7 @@ class RANSAC(nn.Module):size; None assumes ``iteration`` batches of ``batch_size``.Returns:- Tensor of sampled indices with shape :math:`(batch_size, sample_size)`.+ Tensor of sampled indices with shape ``(batch_size, sample_size)``."""device = torch.device("cpu") if device is None else torch.device(device)@@ -536,7 +572,7 @@ class RANSAC(nn.Module):return _DEFAULT_BATCH * self.max_iterdef resolve_batch_size(self, num_tc: int, device: torch.device) -> int:- """Return the batch size of a ``local_optimization="dlt"`` call: the configured one, or the ``"auto"`` choice.+ """Return the batch size of a ``local_optimization="dlt"`` call: configured or ``"auto"``.On CUDA and MPS a homography or affine batch costs about the same from a few hundred up to 8192 hypotheses(the four- and three-point solves are launch-bound), so the whole :attr:`sample_budget` is drawn in batches@@ -692,8 +728,8 @@ class RANSAC(nn.Module):"""Estimate models from minimal samples.Args:- kp1: source keypoints with shape :math:`(batch_size, sample_size, 2)`.- kp2: target keypoints with shape :math:`(batch_size, sample_size, 2)`.+ kp1: source keypoints with shape ``(batch_size, sample_size, 2)``.+ kp2: target keypoints with shape ``(batch_size, sample_size, 2)``.Returns:Estimated models tensor.diff --git a/kornia/geometry/solvers/__init__.py b/kornia/geometry/solvers/__init__.pyindex 561fd6590..b609aa7a5 100644--- a/kornia/geometry/solvers/__init__.py+++ b/kornia/geometry/solvers/__init__.py@@ -15,9 +15,22 @@# limitations under the License.#-"""Kornia Geometry Solvers ā Polynomial and geometric solvers for Kornia.+"""Solvers the geometric estimators build on: an affine fit, a null vector, polynomial roots.-This subpackage provides polynomial solvers and related utilities for geometry tasks.+The polynomial solvers take one polynomial per row, ``(B, k + 1)`` coefficients for degree ``k``, highest degree+first, and return its ``k`` real roots ``(B, k)``, with a repeated root repeated and ``0.0`` in place of a missing+real root; a zero leading coefficient lowers the degree. The polynomial products of Nister's five-point solver use+the monomial orders their docstrings give. :ref:`Two-view geometry <two-view-conventions>` compares the root layout+with ``numpy.roots``.++Which function to use:++- A 2D affine transformation from point correspondences, by weighted least squares: :func:`find_affine`; with+ outliers, :class:`~kornia.geometry.ransac.RANSAC` with ``model_type="affine"``.+- The null vector of a rank-3 ``(*, 3, 4)`` matrix without an SVD: :func:`null_vector_3x4`.+- The real roots of polynomials: :func:`solve_quadratic`, :func:`solve_cubic`, :func:`solve_quartic`.+- Steps of Nister's five-point solver, which :func:`~kornia.geometry.epipolar.find_essential` runs whole:+ :func:`multiply_deg_one_poly`, :func:`multiply_deg_two_one_poly` and :func:`determinant_to_polynomial`."""from .affine import find_affinediff --git a/kornia/geometry/solvers/affine.py b/kornia/geometry/solvers/affine.pyindex 310376e73..204e99fd9 100644--- a/kornia/geometry/solvers/affine.py+++ b/kornia/geometry/solvers/affine.py@@ -15,7 +15,7 @@# limitations under the License.#-"""Affine transformations estimated from point correspondences."""+"""Fit 2D affine transformations to point correspondences by weighted least squares."""from __future__ import annotations@@ -189,13 +189,19 @@ def _find_affine_triplet(def find_affine(points1: torch.Tensor, points2: torch.Tensor, weights: Optional[torch.Tensor] = None) -> torch.Tensor:- r"""Estimate 2D affine transformations from point correspondences.+ r"""Fit a 2D affine transformation to point correspondences by weighted least squares.- Fits the weighted least-squares model :math:`\mathbf{y}=A\mathbf{x}+\mathbf{t}`- independently for each batch. The calculation centers and isotropically normalizes- each point set before solving its 2-by-2 normal equation. Inputs must have the- same floating-point dtype and device; half inputs are accumulated in float32.- Ambient autocast is disabled during fitting to preserve this arithmetic precision.+ ``M = find_affine(points1, points2)`` takes source and destination points ``(B, N, 2)``, ``N >= 3``, and+ optional weights ``(B, N)``, and returns the homogeneous source-to-destination matrices ``(B, 3, 3)``, with bottom+ row ``[0, 0, 1]``, of the least-squares fit :math:`\mathbf{y}=A\mathbf{x}+\mathbf{t}`, independently for each+ batch element. ``M[:, :2]`` is the ``(B, 2, 3)`` matrix that :func:`~kornia.geometry.transform.warp_affine`+ takes. With outliers, use :class:`~kornia.geometry.ransac.RANSAC` with ``model_type="affine"``; for a perspective+ map, :func:`~kornia.geometry.homography.find_homography_dlt`; to build an affine matrix from an angle, a scale and+ a translation instead, :func:`~kornia.geometry.transform.get_affine_matrix2d`.++ The calculation centers and isotropically normalizes each point set before solving its 2-by-2 normal equation.+ Inputs must have the same floating-point dtype and device; half inputs are accumulated in float32. Ambient+ autocast is disabled during fitting to preserve this arithmetic precision.Args:points1: Source points with shape ``(B, N, 2)``, where :math:`N \geq 3`.@@ -208,6 +214,17 @@ def find_affine(points1: torch.Tensor, points2: torch.Tensor, weights: Optional[Homogeneous affine transformations with shape ``(B, 3, 3)``. Degeneratebatches without three non-collinear, positive-weight source points containNaNs, allowing robust estimators to reject singular fits as invalid models.++ Raises:+ ValueError: if the points are not ``(B, N, 2)`` with ``N >= 3``, if their shapes or devices differ, or if+ ``weights`` is not ``(B, N)`` on the points' device.+ TypeError: if the points or ``weights`` are not floating point, or the dtypes of the points differ.++ See Also:+ - :func:`~kornia.geometry.homography.find_homography_dlt`: a perspective map, the general case with eight+ degrees of freedom.+ - :func:`~kornia.geometry.transform.get_affine_matrix2d`: an affine matrix built from parameters, not fitted.+ - :class:`~kornia.geometry.ransac.RANSAC`: an affine map robust to wrong correspondences."""_check_points(points1, points2, 3, batched=True)weights = _prepare_weights(weights, points1, points1.shape[:-2])diff --git a/kornia/geometry/solvers/homogeneous.py b/kornia/geometry/solvers/homogeneous.pyindex d0f0072bc..b23a94d85 100644--- a/kornia/geometry/solvers/homogeneous.py+++ b/kornia/geometry/solvers/homogeneous.py@@ -63,7 +63,12 @@ def _det3(def null_vector_3x4(A: torch.Tensor) -> torch.Tensor:- r"""Return the null vector of a rank-3 matrix of shape ``(*, 3, 4)``.+ r"""Compute the null vector of rank-3 ``(*, 3, 4)`` matrices from cofactors, without an SVD.++ ``v = null_vector_3x4(A)`` takes matrices ``(*, 3, 4)`` and returns unnormalized vectors ``(*, 4)`` with+ :math:`A v = 0`: the last right singular vector up to scale and sign when ``A`` has rank 3, and the zero vector+ when its rank is lower. Each component is a signed ``3 x 3`` determinant, so no LAPACK or cuSOLVER call is made.+ For a least-squares null vector of a matrix with more rows, use an SVD (:func:`torch.linalg.svd`).The null vector :math:`\mathbf{v} \in \mathbb{R}^4` satisfies:math:`A\,\mathbf{v} = \mathbf{0}`. For a matrix of rank 3 this solution@@ -105,6 +110,10 @@ def null_vector_3x4(A: torch.Tensor) -> torch.Tensor:TypeCheckError: if ``A`` is not a :class:`torch.Tensor`.ShapeError: if the last two dimensions of ``A`` are not ``(3, 4)``.+ See Also:+ - :func:`~kornia.geometry.epipolar.triangulate_points`: its ``"cofactor"`` solver uses these minors on the+ ``4 x 4`` DLT systems of a triangulation.+Example:>>> A = torch.tensor([[[1., 0., 0., 0.],... [0., 1., 0., 0.],diff --git a/kornia/geometry/solvers/polynomial_solver.py b/kornia/geometry/solvers/polynomial_solver.pyindex 8ea280cce..8100568eb 100644--- a/kornia/geometry/solvers/polynomial_solver.py+++ b/kornia/geometry/solvers/polynomial_solver.py@@ -15,7 +15,7 @@# limitations under the License.#-"""nn.Module containing the functionalities for computing the real roots of polynomial equation."""+"""Real roots of quadratics, cubics and quartics, and the polynomial algebra of the 5-point solver."""import mathfrom typing import NamedTuple, Tuple@@ -28,9 +28,12 @@ from kornia.core.utils import is_exporting# Reference : https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/polynom_solver.cppdef solve_quadratic(coeffs: torch.Tensor) -> torch.Tensor:- r"""Solve given quadratic equation.+ r"""Solve batched quadratic equations for their real roots, with zeros in place of missing ones.- The function takes the coefficients of quadratic equation and returns the real roots.+ ``roots = solve_quadratic(coeffs)`` takes one quadratic per row, coefficients ``(B, 3)`` highest degree first,+ and returns its two real roots ``(B, 2)``: a repeated root twice, and ``0.0`` in each slot without a real root,+ so a row with complex roots returns ``[0.0, 0.0]``. For degree three or four, use :func:`solve_cubic` or+ :func:`solve_quartic`... math:: coeffs[0]x^2 + coeffs[1]x + coeffs[2] = 0@@ -50,10 +53,14 @@ def solve_quadratic(coeffs: torch.Tensor) -> torch.Tensor:than an on-device solve. The output retains the input dtype and device.Args:- coeffs : The coefficients of quadratic equation :`(B, 3)`+ coeffs : The coefficients of quadratic equation : ``(B, 3)``Returns:- A torch.Tensor of shape `(B, 2)` containing the real roots to the quadratic equation.+ A torch.Tensor of shape ``(B, 2)`` containing the real roots to the quadratic equation.++ See Also:+ - :func:`solve_cubic`: the real roots of cubics, ``(B, 4)`` to ``(B, 3)``.+ - :func:`solve_quartic`: the real roots of quartics, ``(B, 5)`` to ``(B, 4)``.Example:>>> coeffs = torch.tensor([[1., 4., 4.]])@@ -218,10 +225,12 @@ def _scaled_quadratic_coefficients(coeffs: torch.Tensor) -> tuple[torch.Tensor,def solve_cubic(coeffs: torch.Tensor) -> torch.Tensor:- r"""Solve given cubic equation.+ r"""Solve batched cubic equations for their real roots, with zeros in place of missing ones.- The function takes the coefficients of cubic equation and returns- the real roots.+ ``roots = solve_cubic(coeffs)`` takes one cubic per row, coefficients ``(B, 4)`` highest degree first, and+ returns its three real roots ``(B, 3)``, unsorted: a repeated root as often as it repeats, and ``0.0`` in each+ slot without a real root, after the real ones. For degree two or four, use :func:`solve_quadratic` or+ :func:`solve_quartic`... math:: coeffs[0]x^3 + coeffs[1]x^2 + coeffs[2]x + coeffs[3] = 0@@ -239,10 +248,14 @@ def solve_cubic(coeffs: torch.Tensor) -> torch.Tensor:``5 * 10**5`` rows this is slower than an on-device solve. The output retains the input dtype and device.Args:- coeffs : The coefficients cubic equation : `(B, 4)`+ coeffs : The coefficients cubic equation : ``(B, 4)``Returns:- A torch.Tensor of shape `(B, 3)` containing the real roots to the cubic equation.+ A torch.Tensor of shape ``(B, 3)`` containing the real roots to the cubic equation.++ See Also:+ - :func:`solve_quadratic`: the real roots of quadratics, ``(B, 3)`` to ``(B, 2)``.+ - :func:`solve_quartic`: the real roots of quartics, ``(B, 5)`` to ``(B, 4)``.Example:>>> solve_cubic(torch.tensor([[1., 0., 0., 1.]]))@@ -893,10 +906,13 @@ def _quartic_certified_genuine(coeffs: torch.Tensor, *rest: torch.Tensor) -> tupdef solve_quartic(coeffs: torch.Tensor) -> torch.Tensor:- r"""Solve given quartic equation.+ r"""Solve batched quartic equations for their real roots, with zeros in place of missing ones.- The function takes the coefficients of quartic equation and returns- the real roots.+ ``roots = solve_quartic(coeffs)`` takes one quartic per row, coefficients ``(B, 5)`` highest degree first, and+ returns its four real roots ``(B, 4)``, in descending order: a repeated root as often as it repeats, and ``0.0``+ in each slot without a real root, after the real ones. A row that is solved as a cubic, as described below,+ keeps the order of :func:`solve_cubic` instead. For degree two or three, use :func:`solve_quadratic` or+ :func:`solve_cubic`... math:: coeffs[0]x^4 + coeffs[1]x^3 + coeffs[2]x^2 + coeffs[3]x + coeffs[4] = 0@@ -916,10 +932,15 @@ def solve_quartic(coeffs: torch.Tensor) -> torch.Tensor:quartic at every scale; the bound can only move a row from the cubic path to the quartic path.Args:- coeffs : The coefficients quartic equation : `(B, 5)`+ coeffs : The coefficients quartic equation : ``(B, 5)``Returns:- A torch.Tensor of shape `(B, 4)` containing the real roots to the quartic equation.+ A torch.Tensor of shape ``(B, 4)`` containing the real roots to the quartic equation.++ See Also:+ - :func:`solve_quadratic`: the real roots of quadratics, ``(B, 3)`` to ``(B, 2)``.+ - :func:`solve_cubic`: the real roots of cubics, ``(B, 4)`` to ``(B, 3)``, which rows of a lower degree+ are solved as.Example:>>> coeffs = torch.tensor([[1., -10., 35., -50., 24.]])@@ -1348,7 +1369,12 @@ T_deg1[15, 9] = 1 # 1 * 1def multiply_deg_one_poly(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:- r"""Multiply two polynomials of the first order [@nister2004efficient].+ r"""Multiply two batched polynomials of degree one in x, y and z into one of degree two.++ ``c = multiply_deg_one_poly(a, b)`` takes the coefficients ``(B, 4)`` of two linear polynomials in the monomials+ :math:`(x, y, z, 1)` and returns the coefficients ``(B, 10)`` of their product, in the monomial order given+ under Returns. It is a step of Nister's five-point solver [@nister2004efficient]; to estimate essential matrices,+ call :func:`~kornia.geometry.epipolar.find_essential`.Args:a: a first order polynomial for variables :math:`(x,y,z,1)`.@@ -1357,6 +1383,9 @@ def multiply_deg_one_poly(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:Returns:degree 2 poly with the order :math:`(x^2, x*y, x*z, x, y^2, y*z, y, z^2, z, 1)`.+ See Also:+ - :func:`multiply_deg_two_one_poly`: the next step, a degree-two polynomial times a degree-one one.+"""global T_deg1 # noqa: PLW0603if T_deg1.device != a.device or T_deg1.dtype != a.dtype:@@ -1412,7 +1441,13 @@ T_deg2[39, 19] = 1 # (9*4+3)def multiply_deg_two_one_poly(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor:- r"""Multiply two polynomials a and b of degrees two and one [@nister2004efficient].+ r"""Multiply batched polynomials of degree two and one in x, y and z into one of degree three.++ ``c = multiply_deg_two_one_poly(a, b)`` takes the coefficients ``(B, 10)`` of quadratics in the monomial order of+ :func:`multiply_deg_one_poly`'s result and ``(B, 4)`` of linear polynomials in :math:`(x, y, z, 1)`, and returns+ the coefficients ``(B, 20)`` of their product, in the monomial order given under Returns. It is a step of+ Nister's five-point solver [@nister2004efficient]; to estimate essential matrices, call+ :func:`~kornia.geometry.epipolar.find_essential`.Args:a: a second degree poly for variables :math:`(x^2, x*y, x*z, x, y^2, y*z, y, z^2, z, 1)`.@@ -1423,6 +1458,11 @@ def multiply_deg_two_one_poly(a: torch.Tensor, b: torch.Tensor) -> torch.Tensor::math:`(x^3, y^3, x^2*y, x*y^2, x^2*z, x^2, y^2*z, y^2,x*y*z, x*y, x*z^2, x*z, x, y*z^2, y*z, y, z^3, z^2, z, 1)`.+ See Also:+ - :func:`multiply_deg_one_poly`: the product of two degree-one polynomials, whose result this takes as+ ``a``.+ - :func:`determinant_to_polynomial`: the step that follows the elimination of these cubics.+"""global T_deg2 # noqa: PLW0603if T_deg2.device != a.device or T_deg2.dtype != a.dtype:@@ -2900,7 +2940,14 @@ coefficient_map = torch.tensor(def determinant_to_polynomial(A: torch.Tensor,) -> torch.Tensor:- r"""Represent the determinant by the 10th polynomial, used for 5PC solver [@nister2004efficient].+ r"""Expand the determinant of Nister's hidden-variable matrix into a degree-ten polynomial in z.++ ``cs = determinant_to_polynomial(A)`` takes matrices ``(B, 3, 13)`` whose rows hold three polynomials in ``z``,+ the entries of a ``3 x 3`` matrix, and returns the coefficients ``(B, 11)`` of its determinant, lowest degree+ first: the degree-ten polynomial whose real roots give the candidates of Nister's five-point solver+ [@nister2004efficient]. To estimate essential matrices, call :func:`~kornia.geometry.epipolar.find_essential`;+ to find the roots of a quadratic, cubic or quartic, :func:`solve_quadratic`, :func:`solve_cubic` or+ :func:`solve_quartic`.Convention:- Each row of ``A`` holds two cubics (columns 0 to 3 and 4 to 7) and a quartic (columns 8 to 12) in@@ -2913,6 +2960,10 @@ def determinant_to_polynomial(Returns:a degree 10 poly of shape ``(B, 11)``, representing determinant (Eqn. 14 in the paper).+ See Also:+ - :func:`multiply_deg_two_one_poly`: the cubic constraints that this matrix is eliminated from.+ - :func:`~kornia.geometry.epipolar.find_essential`: the whole five-point solver.+"""B, device, dtype = A.shape[0], A.device, A.dtypeglobal multiplication_indices, signs, coefficient_map # noqa: PLW0603diff --git a/kornia/geometry/transform/__init__.py b/kornia/geometry/transform/__init__.pyindex c1d46df49..0e965614b 100644--- a/kornia/geometry/transform/__init__.py+++ b/kornia/geometry/transform/__init__.py@@ -15,9 +15,40 @@# limitations under the License.#-"""Kornia Geometry Transform ā Geometric transformation operations for Kornia.+"""Warp, resize, crop, flip and register images and volumes, with their matrices and pyramids.-This subpackage provides affine, crop, and other geometric transformation utilities.+Images are channels first, ``(B, C, H, W)``, and volumes ``(B, C, D, H, W)``. :func:`warp_perspective`,+:func:`warp_affine` and their 3D forms, the transforms by parameters and the resampling crops take or build matrices+that map **source** pixels to **destination** pixels, with pixel centres at integers and ``x`` the column. The+homography warps (:func:`homography_warp`, :func:`homography_warp3d`, :class:`HomographyWarper`) and the image+registrator default to a destinationāsource homography in normalized :math:`[-1, 1]` coordinates instead, and the+grid warps (:func:`warp_grid`, :func:`warp_grid3d`) take a destinationāsource one in the coordinates of the grid.+``align_corners`` defaults differ between functions; see :doc:`Conventions & Pitfalls </get-started/conventions>`.+Every name below is exported here; the submodules in parentheses group them.++Which function to use:++- Warp by a matrix to an output size (``imgwarp``): :func:`warp_perspective`, :func:`warp_affine`, their 3D forms+ :func:`warp_perspective3d` and :func:`warp_affine3d`; by a normalized destinationāsource homography,+ :func:`homography_warp` (``imgwarp``) and its module form :class:`HomographyWarper` (``homography_warper``); at+ per-pixel coordinates, :func:`remap` (``imgwarp``).+- Build the matrices (``imgwarp``): :func:`get_perspective_transform` from four point pairs,+ :func:`get_rotation_matrix2d`, :func:`get_affine_matrix2d` and their 3D counterparts,+ :func:`get_projective_transform` and :func:`get_affine_matrix3d`, :func:`invert_affine_transform`.+- Rotate, shift, zoom, shear or warp by a matrix at the image size (``affwarp``): :func:`rotate`, :func:`translate`,+ :func:`scale`, :func:`shear`, :func:`affine`, their module forms, and :func:`rotate3d`, :func:`affine3d` for volumes.+- Change the size (``affwarp``): :func:`resize`, :func:`rescale`, :func:`resize_to_be_divisible`.+- Crop (``crop2d``, ``crop3d``): :func:`crop_by_indices` by integer slicing, :func:`crop_and_resize` and+ :func:`crop_by_boxes` by boxes, :func:`crop_by_transform_mat` by a matrix, :func:`center_crop` and+ :class:`CenterCrop2D`, and :func:`crop_and_resize3d` and the other ``3d`` crops for volumes.+- Flip (``flips``): :func:`hflip`, :func:`vflip`, :func:`rot180` and their module forms.+- Pyramids (``pyramid``): :func:`pyrdown`, :func:`pyrup`, :func:`build_pyramid`, :func:`build_laplacian_pyramid`, and+ :class:`ScalePyramid` for scale-space detection.+- Thin-plate splines (``thin_plate_spline``): :func:`get_tps_transform`, :func:`warp_points_tps`,+ :func:`warp_image_tps`.+- Elastic deformation by a smoothed displacement field (``elastic_transform``): :func:`elastic_transform2d`.+- Register two images by optimizing a homography or a similarity (``image_registrator``):+ :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator`."""from .affwarp import *diff --git a/kornia/geometry/transform/affwarp.py b/kornia/geometry/transform/affwarp.pyindex 508122536..a2fedc5f0 100644--- a/kornia/geometry/transform/affwarp.py+++ b/kornia/geometry/transform/affwarp.py@@ -15,6 +15,30 @@# limitations under the License.#+"""Rotate, shift, zoom, shear or resize images, and warp them by an affine matrix at their size.++The affine functions take ``(C, H, W)`` or ``(B, C, H, W)`` images, or ``(C, D, H, W)`` and ``(B, C, D, H, W)``+volumes for the ``3d`` forms, keep the input size and warp with :func:`~kornia.geometry.transform.warp_affine` (or+:func:`~kornia.geometry.transform.warp_affine3d`); all but ``affine`` and ``affine3d`` build the sourceādestination+pixel matrix themselves. Centres and shifts are pixels in ``(x, y)`` (or ``(x, y, z)``) order, rotation angles are+degrees (the shear angles of :class:`Affine` are radians), and a positive 2D angle turns counter-clockwise on+screen. The resizes take ``(*, H, W)`` and change the size with :func:`torch.nn.functional.interpolate`. A module+form stores its tensors as buffers, or as parameters when given a ``torch.nn.Parameter``, so they follow+``Module.to``.++Which function to use:++- Warp by an affine matrix at the input size: :func:`affine` ``(B, 2, 3)``, :func:`affine3d` ``(B, 3, 4)``.+ :class:`Affine` builds the 2D matrix from an angle, a translation, a scale and a shear in one module.+- Rotate about the centre: :func:`rotate` and :class:`Rotate`; a volume by an axis-angle vector: :func:`rotate3d`.+- Shift by pixels: :func:`translate` and :class:`Translate`. Zoom about the centre at the same size: :func:`scale`+ and :class:`Scale`. Shear by raw factors: :func:`shear` and :class:`Shear`.+- Change the size: to ``(h, w)`` or one side's length, :func:`resize` and :class:`Resize`; by a factor,+ :func:`rescale` and :class:`Rescale`; to the nearest multiples of a factor, :func:`resize_to_be_divisible`.+- For an output size other than the input's, warp by a matrix with :func:`~kornia.geometry.transform.warp_affine`+ or :func:`~kornia.geometry.transform.warp_perspective`.+"""+import numbersfrom typing import Optional, Tuple, Union@@ -156,7 +180,13 @@ def affine(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Apply an affine transformation to the image.+ """Warp an image by a sourceādestination affine matrix, keeping its size.++ ``out = affine(tensor, matrix)`` warps ``(C, H, W)`` or ``(B, C, H, W)`` images by ``(B, 2, 3)`` sourceādestination+ pixel affine matrices and returns them at their own size: :func:`warp_affine` with ``dsize=(H, W)``, plus an+ unbatched image and broadcasting. One matrix serves every image of a batch, and one image, unbatched or of batch 1,+ is warped by every matrix, so it returns ``(B, C, H, W)`` for ``B > 1`` matrices. For another output size, use+ :func:`warp_affine`; to build the matrix from an angle, a shift, a scale and a shear, use :class:`Affine`... image:: _static/img/warp_affine.png@@ -169,14 +199,21 @@ def affine(Args:tensor: The image tensor to be warped in shapes of``(C, H, W)`` or ``(B, C, H, W)``.- matrix: The 2x3 affine transformation matrix.+ matrix: The 2x3 affine transformation matrices, with shape ``(B, 2, 3)``.mode: interpolation mode to calculate output values ``'bilinear'`` | ``'nearest'``.padding_mode: padding mode for outside grid values``'zeros'`` | ``'border'`` | ``'reflection'``.align_corners: interpolation flag.Returns:- The warped image with the same shape as the input.+ The warped image with the same shape as the input, except that one image warped by ``B > 1`` matrices+ returns ``(B, C, H, W)``.++ See Also:+ - :func:`warp_affine`: the same warp with an explicit output size, for batched images.+ - :func:`affine3d`: the same for volumes, by ``(B, 3, 4)`` matrices.+ - :class:`Affine`: a module built from an angle, a translation, a scale and a shear instead of a matrix.+ - :func:`rotate`: rotate about the centre without building the matrix.Example:>>> img = torch.rand(1, 2, 3, 5)@@ -220,7 +257,14 @@ def affine3d(padding_mode: str = "zeros",align_corners: bool = False,) -> torch.Tensor:- r"""Apply an affine transformation to the 3d volume.+ """Warp a volume by a sourceādestination 3x4 affine matrix, keeping its size.++ ``out = affine3d(tensor, matrix)`` warps ``(C, D, H, W)`` or ``(B, C, D, H, W)`` volumes by ``(B, 3, 4)``+ sourceādestination voxel affine matrices and returns them at their own size, as :func:`warp_affine3d` with+ ``dsize=(D, H, W)`` does; one matrix serves every volume of a batch. Pass ``align_corners=True``: at the default+ ``False`` the warp mixes two voxel normalizations, so a translation of ``t`` voxels moves the volume+ ``t * size / (size - 1)`` voxels along that axis (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For+ images, use :func:`affine`.Convention:- ``matrix`` is the sourceādestination **pixel** affine matrix ``(B, 3, 4)``@@ -233,11 +277,16 @@ def affine3d(mode: interpolation mode to calculate output values``'bilinear'`` | ``'nearest'``.padding_mode: padding mode for outside grid values- `` 'zeros'`` | ``'border'`` | ``'reflection'``.+ ``'zeros'`` | ``'border'`` | ``'reflection'``.align_corners: interpolation flag.Returns:- The warped image.+ The warped image, with the shape of the input.++ See Also:+ - :func:`affine`: the 2D warp of images, whose default ``align_corners=True`` is correct.+ - :func:`warp_affine3d`: the same warp with an explicit output size.+ - :func:`rotate3d`: rotate a volume about its centre without building the matrix.Example:>>> img = torch.rand(1, 2, 4, 3, 5)@@ -281,7 +330,14 @@ def rotate(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Rotate the tensor anti-clockwise about the center.+ """Rotate an image counter-clockwise by an angle in degrees about its centre or a given point.++ ``out = rotate(tensor, angle)`` rotates ``(C, H, W)`` or ``(B, C, H, W)`` images by ``angle`` degrees, ``(B,)`` or+ one value for all, counter-clockwise as displayed, about ``center`` (``(B, 2)`` ``(x, y)`` pixels, by default the+ image centre ``((W - 1) / 2, (H - 1) / 2)``), and returns them at their size; the uncovered corners are padded+ by ``padding_mode``, except along a one-pixel axis, which the default ``align_corners=True`` repeats+ (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). For a stored angle, use :class:`Rotate`; for volumes,+ :func:`rotate3d`; for the matrix, to compose or reuse it, :func:`get_rotation_matrix2d`... image:: _static/img/rotate.png@@ -305,6 +361,13 @@ def rotate(Returns:The rotated tensor with shape as input.+ See Also:+ - :class:`Rotate`: the module form, with the angle stored.+ - :func:`rotate3d`: rotate volumes by an axis-angle vector; a positive ``z`` angle turns clockwise on screen+ (3D rotations are right-handed by design).+ - :func:`get_rotation_matrix2d`: the ``(B, 2, 3)`` matrix of this rotation, with a scale.+ - :func:`rot180`: a half turn by flipping both axes, without interpolation.+.. note::See a working example `here <https://www.kornia.org/tutorials/nbs/rotate_affine.html>`__.@@ -366,7 +429,16 @@ def rotate3d(padding_mode: str = "zeros",align_corners: bool = False,) -> torch.Tensor:- r"""Rotate 3D the tensor anti-clockwise about the centre.+ """Rotate a volume by an axis-angle vector of yaw, pitch and roll in degrees.++ ``out = rotate3d(tensor, yaw, pitch, roll)`` rotates ``(C, D, H, W)`` or ``(B, C, D, H, W)`` volumes by the+ axis-angle vector ``(yaw, pitch, roll)`` in degrees, ``(B,)`` each, about ``center`` (``(B, 3)`` ``(x, y, z)``+ pixels, by default the volume centre), and returns them at their size. Positive angles follow the right-hand rule,+ so a positive ``roll``, about ``z``, turns clockwise on screen, the opposite of :func:`rotate`+ (3D rotations are right-handed by design). Pass ``align_corners=True``: at the default ``False`` the+ warp mixes two voxel normalizations, which changes the rotation unless it turns about the volume centre in a plane+ whose two sizes are equal, as any rotation of a cube about its centre does+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_).Convention:- ``center`` is ``(x, y, z)`` in pixels, origin at the top-left of the first depth@@ -402,6 +474,11 @@ def rotate3d(Returns:torch.Tensor: The rotated tensor with shape as input.+ See Also:+ - :func:`rotate`: the 2D rotation of images, counter-clockwise on screen.+ - :func:`affine3d`: warp a volume by any ``(B, 3, 4)`` affine matrix.+ - :func:`get_projective_transform`: the ``(B, 3, 4)`` matrix of this rotation, with a scale.+"""if not isinstance(tensor, torch.Tensor):raise TypeError(f"Input tensor type is not a torch.Tensor. Got {type(tensor)}")@@ -445,7 +522,15 @@ def translate(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Translate the tensor in pixel units.+ """Shift an image by a translation in pixels.++ ``out = translate(tensor, translation)`` shifts ``(C, H, W)`` or ``(B, C, H, W)`` images by ``translation``+ ``(B, 2)``, ``(dx, dy)`` in pixels, a positive ``dx`` to the right and a positive ``dy`` down, and returns them at+ their size; a fractional shift interpolates, and the uncovered border is padded by ``padding_mode``, except+ along a one-pixel axis, which the default ``align_corners=True`` repeats+ (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). One+ translation serves every image of a batch, and one image, unbatched or of batch 1, is shifted by every+ translation. For a stored translation, use :class:`Translate`; for the matrix, :func:`get_translation_matrix2d`... image:: _static/img/translate.png@@ -453,7 +538,7 @@ def translate(- ``translation`` is ``(dx, dy)`` in pixelsArgs:- tensor: The image tensor to be warped in shapes of ``(B, C, H, W)``.+ tensor: The image tensor to be warped in shapes of ``(C, H, W)`` or ``(B, C, H, W)``.translation: tensor containing the amount of pixels totranslate in the x and y direction. The tensor must have a shape of(B, 2), where B is batch size, last dimension contains dx dy.@@ -464,7 +549,13 @@ def translate(align_corners: interpolation flag.Returns:- The translated tensor with shape as input.+ The translated tensor with shape as input, except that one image shifted by ``B > 1`` translations returns+ ``(B, C, H, W)``.++ See Also:+ - :class:`Translate`: the module form, with the translation stored.+ - :func:`get_translation_matrix2d`: the ``(B, 3, 3)`` matrix of this shift.+ - :func:`affine`: warp by any affine matrix at the input size.Example:>>> img = torch.rand(1, 3, 4, 4)@@ -498,7 +589,14 @@ def scale(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Scale the tensor by a factor.+ """Zoom an image about its centre by a scale factor, keeping its size.++ ``out = scale(tensor, scale_factor)`` scales the content of ``(C, H, W)`` or ``(B, C, H, W)`` images by+ ``scale_factor``, ``(B,)`` for both axes or ``(B, 2)`` as ``(sx, sy)``, about ``center`` (``(B, 2)`` ``(x, y)``+ pixels, by default the image centre), and returns them at their size: a factor above 1 zooms in and crops, one+ below 1 shrinks the content and pads the border, except along a one-pixel axis, which the default+ ``align_corners=True`` repeats (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). To change the image+ size instead, use :func:`rescale` or :func:`resize`; for a stored factor, :class:`Scale`... image:: _static/img/scale.png@@ -523,6 +621,11 @@ def scale(Returns:The scaled tensor with the same shape as the input.+ See Also:+ - :class:`Scale`: the module form, with the factor stored.+ - :func:`rescale`: change the image size by a factor, given height first.+ - :func:`get_rotation_matrix2d`: the matrix of this scaling, with a zero angle.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> scale_factor = torch.tensor([[2., 2.]])@@ -563,7 +666,14 @@ def shear(padding_mode: str = "zeros",align_corners: bool = False,) -> torch.Tensor:- r"""Shear the tensor.+ """Shear an image by raw shear factors about pixel (0, 0).++ ``out = shear(tensor, shear)`` shears ``(C, H, W)`` or ``(B, C, H, W)`` images by ``shear`` ``(B, 2)``, the raw+ factors ``(shx, shy)`` described below, and returns them at their size. One pair of factors serves every image of+ a batch, and one image, unbatched or of batch 1, is sheared by every pair. For shear angles in radians, use+ :class:`Affine`, about the image centre by default, or :func:`get_shear_matrix2d`, about a centre you pass; both+ shear by the negative tangent, so values below ``pi / 2`` in magnitude shear the other way there. For stored+ factors, use :class:`Shear`... image:: _static/img/shear.png@@ -577,7 +687,7 @@ def shear(:class:`Shear`, which default to ``True``)Args:- tensor: The image tensor to be skewed with shape of ``(B, C, H, W)``.+ tensor: The image tensor to be skewed with shape of ``(C, H, W)`` or ``(B, C, H, W)``.shear: tensor containing raw shear factorsin the x and y direction about pixel (0, 0). The tensor must have a shape of(B, 2), where B is batch size, last dimension contains shx shy.@@ -588,7 +698,14 @@ def shear(align_corners: interpolation flag.Returns:- The skewed tensor with shape same as the input.+ The skewed tensor with shape same as the input, except that one image sheared by ``B > 1`` pairs of+ factors returns ``(B, C, H, W)``.++ See Also:+ - :class:`Shear`: the module form, which defaults to ``align_corners=True``.+ - :func:`get_shear_matrix2d`: a shear given as angles in radians about a centre, the reading of+ :class:`Affine`.+ - :func:`affine`: warp by any affine matrix at the input size.Example:>>> img = torch.rand(1, 3, 4, 4)@@ -634,7 +751,14 @@ def resize(side: str = "short",antialias: bool = False,) -> torch.Tensor:- r"""Resize the input torch.Tensor to the given size.+ """Resize an image to a given size, or one side to a given length keeping the aspect ratio.++ ``out = resize(input, size)`` resamples the last two dimensions of ``(*, H, W)`` tensors to ``size = (h, w)`` with+ :func:`torch.nn.functional.interpolate` and returns ``(*, h, w)``. An ``int`` size sets the length of one side and+ keeps the aspect ratio, truncating the other side to an integer: the shorter side by default, or the longer side,+ the height or the width with ``side='long'``, ``'vert'`` or ``'horz'``. ``antialias=True`` blurs with a Gaussian+ before downscaling. To scale by a factor, use :func:`rescale`; to zoom the content and keep the size,+ :func:`scale`... image:: _static/img/resize.png@@ -649,11 +773,10 @@ def resize(- ``side`` resizing preserves aspect ratio using the named sideArgs:- input: The image tensor to be skewed with shape of ``(..., H, W)``.- `...` means there can be any number of dimensions.+ input: The image tensor to resize, with shape ``(*, H, W)``; ``*`` means any number of dimensions.size: Desired output size. If size is a sequence like (h, w),- output size will be matched to this. If size is an int, smaller edge of the image will- be matched to this number. i.e, if height > width, then image will be rescaled+ output size will be matched to this. If size is an int, the side that ``side`` names will be matched to+ this number; with the default ``side='short'``, if height > width, the image will be rescaledto (size * height / width, size)interpolation: algorithm used for upsampling: ``'nearest'`` | ``'nearest-exact'`` | ``'bilinear'`` |``'bicubic'`` | ``'area'``.@@ -666,6 +789,13 @@ def resize(Returns:The resized tensor with the shape as the specified size.+ See Also:+ - :class:`Resize`: the module form, with the size stored.+ - :func:`rescale`: the output size as a factor of the input size.+ - :func:`resize_to_be_divisible`: the nearest size whose sides are multiples of a factor.+ - :func:`scale`: zoom the content about the centre and keep the size.+ - :func:`crop_and_resize`: resamples a box of each image, not the whole image, to a fixed size.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> out = resize(img, (6, 8))@@ -766,7 +896,14 @@ def resize_to_be_divisible(side: str = "short",antialias: bool = False,) -> torch.Tensor:- """Resize the input tensor to be divisible by a certain factor.+ """Resize an image so that its height and width are multiples of a factor.++ ``out = resize_to_be_divisible(input, divisible_factor)`` resizes ``(*, H, W)`` tensors with :func:`resize` to the+ nearest multiples of ``divisible_factor``, at least ``divisible_factor`` for a positive side, and returns+ ``(*, h, w)``; an empty side stays empty. A side halfway between two multiples rounds as Python's :func:`round`+ does, to an even quotient, so with a factor of 4 both 6 and 10 become 8. ``side`` is passed to :func:`resize` but+ has no effect there, because the size is a pair (`#6397 <https://github.com/kornia/kornia/issues/6397>`_). For a+ given size, use :func:`resize`.Convention:- see the convention block of :func:`resize`@@ -780,13 +917,22 @@ def resize_to_be_divisible(interpolation (str, optional): Interpolation flag. Defaults to "bilinear".align_corners (Optional[bool], optional):whether to align the corners of the input and output. Defaults to None.- side (str, optional): Side to resize. Defaults to "short".+ side (str, optional): passed to :func:`resize`, where it has no effect because the size is a pair.+ Defaults to "short".antialias (bool, optional):If True, then image will be filtered with Gaussian before downscaling. Defaults to False.Returns:torch.Tensor: The resized tensor.+ Raises:+ TypeError: If ``input`` is not a tensor.+ ValueError: If ``input`` has fewer than two dimensions, or ``divisible_factor`` is not a positive integer.++ See Also:+ - :func:`resize`: resize to a given size or side length.+ - :func:`rescale`: resize by a factor.+"""if not isinstance(input, torch.Tensor):raise TypeError(f"Input tensor type is not a torch.Tensor. Got {type(input)}")@@ -808,7 +954,13 @@ def rescale(align_corners: Optional[bool] = None,antialias: bool = False,) -> torch.Tensor:- r"""Rescale the input torch.Tensor with the given factor.+ """Resize an image by a scale factor per side, given height first.++ ``out = rescale(input, factor)`` resizes ``(*, H, W)`` tensors with :func:`resize` to+ ``(int(H * factor_h), int(W * factor_w))``, truncating, and returns ``(*, h, w)``. ``factor`` is one ``float`` for+ both sides or a pair ``(factor_h, factor_w)``, height first, unlike the ``(sx, sy)`` of :func:`scale`; a single+ ``int`` raises ``TypeError``. To set the output size directly, use :func:`resize`; for a stored factor,+ :class:`Rescale`... image:: _static/img/rescale.png@@ -822,7 +974,7 @@ def rescale(when ``factor`` is not a power of two (``0.7`` at a side of 90 gives 63, eager 62)Args:- input: The image tensor to be scale with shape of ``(B, C, H, W)``.+ input: The image tensor to be scale with shape of ``(*, H, W)``.factor: Desired scaling factor as ``(factor_h, factor_w)`` ā height first. If a singlefloat, the value is used for both height and width (an ``int`` raises ``TypeError``).interpolation: algorithm used for upsampling: ``'nearest'`` | ``'nearest-exact'`` | ``'bilinear'`` |@@ -834,6 +986,11 @@ def rescale(Returns:The rescaled tensor with the shape as the specified size.+ See Also:+ - :class:`Rescale`: the module form, which defaults to ``align_corners=True``.+ - :func:`resize`: the output size instead of a factor.+ - :func:`scale`: zoom the content by a factor and keep the size.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> out = rescale(img, (2, 3))@@ -871,7 +1028,11 @@ def rescale(class Resize(nn.Module):- r"""Resize the input torch.Tensor to the given size.+ """Resize an image to a given size, or one side to a given length (module form).++ Build ``resizer = Resize(size)`` with the arguments of :func:`resize`, then ``out = resizer(input)`` resizes+ ``(*, H, W)`` tensors exactly as ``resize(input, size, ...)`` does. For a factor instead of a size, use+ :class:`Rescale`; inside an augmentation pipeline, use :class:`~kornia.augmentation.Resize`.Convention:See :doc:`Conventions & Pitfalls </get-started/conventions>` for sampling and ``align_corners``@@ -882,8 +1043,8 @@ class Resize(nn.Module):Args:size: Desired output size. If size is a sequence like (h, w),- output size will be matched to this. If size is an int, smaller edge of the image will- be matched to this number. i.e, if height > width, then image will be rescaled+ output size will be matched to this. If size is an int, the side that ``side`` names will be matched to+ this number; with the default ``side='short'``, if height > width, the image will be rescaledto (size * height / width, size)interpolation: algorithm used for upsampling: ``'nearest'`` | ``'nearest-exact'`` | ``'bilinear'`` |``'bicubic'`` | ``'area'``.@@ -896,6 +1057,11 @@ class Resize(nn.Module):Returns:The resized tensor with the shape of the given size.+ See Also:+ - :func:`resize`: the one-call form.+ - :class:`Rescale`: the module form of :func:`rescale`, a factor instead of a size.+ - :class:`~kornia.augmentation.Resize`: the augmentation of the same name, for augmentation pipelines.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> out = Resize((6, 8))(img)@@ -944,7 +1110,15 @@ class Resize(nn.Module):class Affine(nn.Module):- r"""Apply multiple elementary affine transforms simultaneously.+ """Warp an image by a rotation, a scale and a shear about a centre, then a translation (module form).++ Build ``warper = Affine(angle, translation, scale_factor, shear, center)`` with at least one of the first four,+ then ``out = warper(input)`` warps ``(B, C, H, W)`` images, or one ``(C, H, W)`` image, by the matrix of+ :func:`get_affine_matrix2d`: a shear, a scale, a counter-clockwise rotation in degrees and a translation in pixels,+ the first three about ``center`` (by default the image centre). The parameters, and ``center`` when given, share+ one batch size; without ``center``, it must equal the input's, a ``(C, H, W)`` input counting as 1+ (`#6395 <https://github.com/kornia/kornia/issues/6395>`_). For a matrix you already have, use :func:`affine`;+ for one kind of warp, :class:`Rotate`, :class:`Translate`, :class:`Scale` or :class:`Shear`.Convention:- ``shear`` contains angles in radians, using the negative-tangent, x-then-y convention@@ -974,13 +1148,27 @@ class Affine(nn.Module):align_corners: interpolation flag.Raises:- RuntimeError: If not one of ``angle``, ``translation``, ``scale_factor``, or ``shear`` is set.+ RuntimeError: At construction, if not one of ``angle``, ``translation``, ``scale_factor``, or ``shear`` is+ set, or if their batch sizes differ. At the call with ``center`` given, if the parameter batch and the+ input batch differ and neither is 1.DeviceError: If ``angle``, ``translation``, ``scale_factor`` and ``shear`` are not all on one device.TypeCheckError: If they are on one device but do not all have one dtype. The ones not given arecreated on that device and in that dtype.+ ValueError: At the call, if ``center`` and the parameters have different batch sizes (construction does not+ check ``center``), or, without ``center``, if the parameter batch differs from the input batch, a+ ``(C, H, W)`` input counting as 1 (`#6395 <https://github.com/kornia/kornia/issues/6395>`_).Returns:- The transformed tensor with same shape as input.+ The transformed tensor with same shape as input, except that with ``center`` given, one image warped by+ ``B > 1`` parameter sets returns ``(B, C, H, W)``.++ See Also:+ - :func:`get_affine_matrix2d`: the ``(B, 3, 3)`` matrix this module warps by, whose own angle turns clockwise+ (`#4408 <https://github.com/kornia/kornia/issues/4408>`_).+ - :func:`affine`: warp by an affine matrix you already have.+ - :class:`Rotate`: a rotation alone, which, unlike this module without ``center``, shares one angle across a+ batch.+ - :class:`Shear`: raw shear factors about pixel ``(0, 0)``, which shear the other way below ``pi / 2``.Example:>>> img = torch.rand(1, 2, 3, 5)@@ -1066,7 +1254,11 @@ class Affine(nn.Module):class Rescale(nn.Module):- r"""Rescale the input torch.Tensor with the given factor.+ """Resize an image by a stored scale factor per side, given height first (module form).++ Build ``rescaler = Rescale(factor)``, then ``out = rescaler(input)`` resizes ``(*, H, W)`` tensors as+ :func:`rescale` does, except that ``align_corners`` defaults to ``True`` here and to ``None`` in the function. For a+ size instead of a factor, use :class:`Resize`.Convention:- align_corners: ``True`` by default (differs from function :func:`rescale`, whose default@@ -1088,6 +1280,11 @@ class Rescale(nn.Module):Returns:The rescaled tensor with the shape according to the given factor.+ See Also:+ - :func:`rescale`: the one-call form, whose ``align_corners`` defaults to ``None``.+ - :class:`Resize`: the output size instead of a factor.+ - :class:`Scale`: zoom the content by a factor and keep the size.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> out = Rescale((2, 3))(img)@@ -1130,7 +1327,12 @@ class Rescale(nn.Module):class Rotate(nn.Module):- r"""Rotate the tensor anti-clockwise about the centre.+ """Rotate an image counter-clockwise by a stored angle in degrees (module form).++ Build ``rotator = Rotate(angle)``, optionally with ``center``, then ``out = rotator(input)`` rotates ``(C, H, W)``+ or ``(B, C, H, W)`` images as :func:`rotate` does: counter-clockwise as displayed, about the image centre by+ default, at the input size. For volumes, use :func:`rotate3d`; to rotate, shift, scale and shear in one warp,+ :class:`Affine`.Convention:- See the convention block of :func:`rotate`.@@ -1150,6 +1352,11 @@ class Rotate(nn.Module):Returns:The rotated tensor with the same shape as the input.+ See Also:+ - :func:`rotate`: the one-call form.+ - :class:`Affine`: a rotation with a translation, a scale and a shear in one warp.+ - :class:`~kornia.geometry.transform.Rot180`: a half turn by flipping both axes, without interpolation.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> angle = torch.tensor([90.])@@ -1191,7 +1398,11 @@ class Rotate(nn.Module):class Translate(nn.Module):- r"""Translate the tensor in pixel units.+ """Shift an image by a stored translation in pixels (module form).++ Build ``shifter = Translate(translation)``, then ``out = shifter(input)`` shifts ``(C, H, W)`` or ``(B, C, H, W)``+ images by the ``(B, 2)`` translation ``(dx, dy)`` in pixels, as :func:`translate` does. To combine the shift with a+ rotation, a scale and a shear, use :class:`Affine`.Convention:- See the convention block of :func:`translate`.@@ -1207,7 +1418,12 @@ class Translate(nn.Module):align_corners: interpolation flag.Returns:- The translated tensor with the same shape as the input.+ The translated tensor with the same shape as the input, except that one image shifted by ``B > 1``+ translations returns ``(B, C, H, W)``.++ See Also:+ - :func:`translate`: the one-call form.+ - :class:`Affine`: a translation with a rotation, a scale and a shear in one warp.Example:>>> img = torch.rand(1, 3, 4, 4)@@ -1247,7 +1463,11 @@ class Translate(nn.Module):class Scale(nn.Module):- r"""Scale the tensor by a factor.+ """Zoom an image about its centre by a stored scale factor, keeping its size (module form).++ Build ``zoomer = Scale(scale_factor)``, optionally with ``center``, then ``out = zoomer(input)`` scales the content+ of ``(C, H, W)`` or ``(B, C, H, W)`` images as :func:`scale` does, at the input size. To change the image size+ instead, use :class:`Rescale` or :class:`Resize`.Convention:- See the convention block of :func:`scale`.@@ -1269,6 +1489,11 @@ class Scale(nn.Module):Returns:The scaled tensor with the same shape as the input.+ See Also:+ - :func:`scale`: the one-call form.+ - :class:`Rescale`: change the image size by a factor instead.+ - :class:`Affine`: a scale with a rotation, a translation and a shear in one warp.+Example:>>> img = torch.rand(1, 3, 4, 4)>>> scale_factor = torch.tensor([[2., 2.]])@@ -1309,7 +1534,13 @@ class Scale(nn.Module):class Shear(nn.Module):- r"""Shear the tensor.+ """Shear an image by stored raw shear factors about pixel (0, 0) (module form).++ Build ``shearer = Shear(shear)``, then ``out = shearer(input)`` shears ``(C, H, W)`` or ``(B, C, H, W)`` images by+ the ``(B, 2)`` raw factors ``(shx, shy)``, as :func:`shear` does, except that ``align_corners`` defaults to+ ``True`` here and to ``False`` in the function. For shear angles in radians about the image centre, use+ :class:`Affine`, which shears by their negative tangent, so values below ``pi / 2`` in magnitude shear the other+ way there.Convention:- align_corners: ``True`` by default (differs from function :func:`shear`, whose default is@@ -1327,7 +1558,13 @@ class Shear(nn.Module):align_corners: interpolation flag.Returns:- The skewed tensor with the same shape as the input.+ The skewed tensor with the same shape as the input, except that one image sheared by ``B > 1`` pairs of+ factors returns ``(B, C, H, W)``.++ See Also:+ - :func:`shear`: the one-call form, which defaults to ``align_corners=False``.+ - :class:`Affine`: shear angles in radians about the image centre, which shear the other way below+ ``pi / 2``.Example:>>> img = torch.rand(1, 3, 4, 4)diff --git a/kornia/geometry/transform/crop2d.py b/kornia/geometry/transform/crop2d.pyindex cbf60e0bf..66c892263 100644--- a/kornia/geometry/transform/crop2d.py+++ b/kornia/geometry/transform/crop2d.py@@ -15,6 +15,34 @@# limitations under the License.#+"""Crop image regions by integer slicing, by boxes or by a matrix, at a given output size.++The crop functions take channels-first images ``(B, C, H, W)`` and raise for an unbatched one. A box is+``(B, 4, 2)``: four ``(x, y)`` pixel corners, top-left, top-right, bottom-right, bottom-left, with **inclusive**+coordinates, so box ``(1, 1)``..``(2, 2)`` is a 2x2 pixel block, and the corners of an exclusive ``xyxy`` box+(torchvision) select one more row and column (`#3934 <https://github.com/kornia/kornia/issues/3934>`_). Sizes are+``(h, w)``. Except for the integer slicing of :func:`crop_by_indices` and of the default mode of+:class:`CenterCrop2D`, the crops resample by a sourceādestination pixel matrix through+:func:`~kornia.geometry.transform.warp_affine` or :func:`~kornia.geometry.transform.warp_perspective`, into an output+window that starts at destination pixel ``(0, 0)``. For these the matrix is in pixels, so along a source axis longer+than one pixel ``align_corners`` changes only how ``'reflection'`` padding mirrors (about the outer pixel centres+with ``True``, about the outer pixel edges with ``False``) and, in ``'nearest'`` mode, which pixel a sample exactly+halfway between two pixels takes. Along a one-pixel source axis, ``align_corners=True`` repeats that pixel where the+window should be padded (`#6433 <https://github.com/kornia/kornia/issues/6433>`_).++Which function to use:++- An axis-aligned box by integer slicing, without interpolation, resized or padded where the slices differ in size:+ :func:`crop_by_indices`.+- Any four-corner box resampled to a fixed ``(h, w)``: :func:`crop_and_resize`; mapped onto a second box whose size+ sets the output: :func:`crop_by_boxes`.+- A crop given as a sourceādestination pixel matrix, ``(B, 2, 3)`` or ``(B, 3, 3)``: :func:`crop_by_transform_mat`.+- The central window of each image: :func:`center_crop`, or :class:`CenterCrop2D` (module form, slicing by default).+- For volumes, use :func:`~kornia.geometry.transform.crop_and_resize3d` and the other ``3d`` crops; in an+ augmentation pipeline, :class:`~kornia.augmentation.RandomCrop`, :class:`~kornia.augmentation.RandomResizedCrop` or+ :class:`~kornia.augmentation.CenterCrop`.+"""+from typing import Optional, Tuple, Unionimport torch@@ -48,7 +76,15 @@ def crop_and_resize(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Extract crops from 2D images (4D torch.Tensor) and resize given a bounding box.+ r"""Crop a four-corner box from each image and resample it to a fixed output size.++ ``patches = crop_and_resize(input_tensor, boxes, size)`` takes images ``(B, C, H, W)`` and boxes ``(B, 4, 2)``, one+ per image or ``(1, 4, 2)`` for all of them, and returns the ``(B, C, h, w)`` patches for ``size = (h, w)``. The box+ corners land on the corner pixels of the patch: a rectangle is resized corner to corner, any other quadrilateral is+ rectified by the homography between the two, and the patch is differentiable with respect to the corners. A single+ image does not broadcast over a batch of boxes. For an exact integer slice without interpolation, use+ :func:`crop_by_indices`; to place the crop by a second box, :func:`crop_by_boxes`; for volumes,+ :func:`crop_and_resize3d`.Convention:See :doc:`Conventions & Pitfalls </get-started/conventions>` for the ``align_corners``@@ -59,13 +95,17 @@ def crop_and_resize(top-left, top-right, bottom-right, bottom-left; coordinates are**inclusive** pixel positions (box ``(1, 1)``..``(2, 2)`` selects a:math:`2 \times 2` pixel block), origin at top-left+ - a box axis of one pixel is cropped exactly when ``size`` is one pixel on that axis too; one pixel in the box+ but not in ``size``, or the reverse, has no invertible matrix and returns NaN (a meaningless finite value+ with ``mode='nearest'``), and so does a box with no homography to the patch, such as a triangle+ (`#4747 <https://github.com/kornia/kornia/issues/4747>`_)Args:- input_tensor: the 2D image torch.Tensor with shape (B, C, H, W).+ input_tensor: the 2D image torch.Tensor with shape ``(B, C, H, W)``.boxes : a torch.Tensor containing the coordinates of the bounding boxes to be extracted.- The torch.Tensor must have the shape of Bx4x2, where each box is defined in the following (clockwise)- order: top-left, top-right, bottom-right and bottom-left. The coordinates must be in the x, y order.- The coordinates would compose a rectangle with a shape of (N1, N2).+ The torch.Tensor must have the shape of ``(B, 4, 2)``, where each box is defined in the following+ (clockwise) order: top-left, top-right, bottom-right and bottom-left. The coordinates must be in the x, y+ order.size: a tuple with the height and width that will beused to resize the extracted patches.mode: interpolation mode to calculate output values@@ -75,7 +115,16 @@ def crop_and_resize(align_corners: mode for grid_generation.Returns:- torch.Tensor: torch.tensor containing the patches with shape BxCxN1xN2.+ the patches, with shape ``(B, C, h, w)`` for ``size = (h, w)``.++ See Also:+ - :func:`crop_by_indices`: integer slicing of axis-aligned boxes, without interpolation.+ - :func:`crop_by_boxes`: maps the box onto a second box, whose size sets the output size.+ - :func:`crop_and_resize3d`: the same for volumes and ``(B, 8, 3)`` boxes; pass ``align_corners=True`` there.+ - :func:`resize`: the whole image; for an image and a ``size`` of at least 2 pixels on each axis, bilinear+ ``crop_and_resize`` of the image's corner box equals ``resize(input, size, align_corners=True)``.+ - :class:`~kornia.augmentation.RandomResizedCrop`: random boxes resized to a fixed size, in an augmentation+ pipeline.Example:>>> input = torch.tensor([[[@@ -162,7 +211,15 @@ def center_crop(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Crop the 2D images (4D torch.Tensor) from the center.+ r"""Crop the central window of each image to a given size, in one call.++ ``patches = center_crop(input_tensor, size)`` takes images ``(B, C, H, W)`` and returns their central+ ``(B, C, h, w)`` windows for ``size = (h, w)``: a sub-image of the input, with the extra pixel of an odd margin at+ the bottom/right. A ``size`` larger than the input pads by ``padding_mode`` (zeros by default), again with the extra+ pixel at the bottom/right, except along a one-pixel input axis, which the default ``align_corners=True`` repeats+ (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). The crop is an integer translation sampled by+ :func:`~kornia.geometry.transform.warp_perspective`, so it is exact up to float rounding. For the module form, which+ slices by default, use :class:`CenterCrop2D`; for volumes, :func:`center_crop3d`.Convention:- input: ``(B, C, H, W)`` (strictly 4D ā no unbatched ``(C, H, W)``/@@ -172,7 +229,7 @@ def center_crop(left at the bottom/right; for a ``size`` larger than the input, the extra padding pixel goes there too.Args:- input_tensor: the 2D image torch.Tensor with shape (B, C, H, W).+ input_tensor: the 2D image torch.Tensor with shape ``(B, C, H, W)``.size: a tuple with the expected height and widthof the output patch.mode: interpolation mode to calculate output values@@ -182,7 +239,14 @@ def center_crop(align_corners: mode for grid_generation.Returns:- the output torch.Tensor with patches.+ the central windows, with shape ``(B, C, h, w)``.++ See Also:+ - :class:`CenterCrop2D`: the module form; slices with integer indices by default instead of resampling.+ - :func:`center_crop3d`: the central window of each volume; keep its default ``align_corners=True``.+ - :func:`crop_and_resize`: a window at any box, resampled to a fixed size.+ - :class:`~kornia.augmentation.CenterCrop`: the same windows as an augmentation, for augmentation pipelines;+ it raises for a ``size`` larger than the input instead of padding.Examples:>>> input = torch.tensor([[[@@ -254,12 +318,17 @@ def crop_by_boxes(align_corners: bool = True,validate_boxes: bool = True,) -> torch.Tensor:- """Perform crop transform on 2D images (4D torch.Tensor) given two bounding boxes.-- Given an input torch.Tensor, this function selected the interested areas by the provided bounding boxes (src_box).- Then the selected areas would be fitted into the targeted bounding boxes (dst_box) by a perspective transformation.- So far, the ragged torch.Tensor is not supported by PyTorch right now. This function hereby requires- the bounding boxes in a batch must be rectangles with same width and height.+ """Crop images by mapping a source box onto a destination box, whose size sets the output size.++ ``patches = crop_by_boxes(input_tensor, src_box, dst_box)`` takes images ``(B, C, H, W)`` and two boxes per image,+ ``(B, 4, 2)`` each (both ``(1, 4, 2)`` for all images), warps each image by the homography that maps ``src_box``+ onto ``dst_box`` and returns the ``(B, C, h, w)`` window at destination pixels ``(0, 0)``..``(w - 1, h - 1)``, where+ ``(h, w)`` is the inclusive size of ``dst_box``. Every ``dst_box`` in the batch must have that same size, since the+ output is one tensor, and a ``dst_box`` that does not start at ``(0, 0)`` shifts the crop by its offset, partly out+ of the window. The size is read from the ``dst_box`` values on the host. When ``dst_box`` is the window+ ``(0, 0)``..``(w - 1, h - 1)`` itself and ``(h, w)`` is known, :func:`crop_and_resize` with ``size=(h, w)`` gives+ the same crop without that read; it always places the destination box at the origin, so it cannot reproduce an+ offset ``dst_box``. For volumes, use :func:`crop_by_boxes3d`.Convention:- input: ``(B, C, H, W)``@@ -267,16 +336,19 @@ def crop_by_boxes(top-left, top-right, bottom-right, bottom-left ā same convention as:func:`crop_and_resize`; ``dst_box`` determines the output resolution- a single box (batch size 1) broadcasts over a batch of images, but a single- image does not broadcast over a batch of boxes+ image does not broadcast over a batch of boxes; ``src_box`` and ``dst_box`` have the same batch size+ - a box axis of one pixel in one box but not in the other returns NaN (a meaningless finite value with+ ``mode='nearest'``), as in :func:`crop_and_resize`+ (`#4747 <https://github.com/kornia/kornia/issues/4747>`_)Args:- input_tensor: the 2D image torch.Tensor with shape (B, C, H, W).- src_box: a torch.Tensor with shape (B, 4, 2) containing the coordinates of the bounding boxes- to be extracted. The torch.Tensor must have the shape of Bx4x2, where each box is defined+ input_tensor: the 2D image torch.Tensor with shape ``(B, C, H, W)``.+ src_box: a torch.Tensor with shape ``(B, 4, 2)`` containing the coordinates of the bounding boxes+ to be extracted, where each box is definedin the clockwise order: top-left, top-right, bottom-right and bottom-left.The coordinates must be in x, y order.- dst_box: a torch.Tensor with shape (B, 4, 2) containing the coordinates of the bounding boxes- to be placed. The torch.Tensor must have the shape of Bx4x2, where each box is defined+ dst_box: a torch.Tensor with shape ``(B, 4, 2)`` containing the coordinates of the bounding boxes+ to be placed, where each box is definedin the clockwise order: top-left, top-right, bottom-right and bottom-left.The coordinates must be in x, y order.mode: interpolation mode to calculate output values@@ -284,10 +356,19 @@ def crop_by_boxes(padding_mode: padding mode for outside grid values``'zeros'`` | ``'border'`` | ``'reflection'``.align_corners: mode for grid_generation.- validate_boxes: flag to perform validation on boxes.+ validate_boxes: has no effect: the boxes are not validated+ (`#6401 <https://github.com/kornia/kornia/issues/6401>`_).Returns:- torch.Tensor: the output torch.tensor with patches.+ the patches, with shape ``(B, C, h, w)`` for the size ``(h, w)`` of ``dst_box``.++ Raises:+ AssertionError: if the boxes of ``dst_box`` differ in size.++ See Also:+ - :func:`crop_and_resize`: takes the output size as ``size`` and places the box at the patch corners.+ - :func:`crop_by_transform_mat`: takes the sourceādestination matrix itself.+ - :func:`crop_by_boxes3d`: the same for volumes and ``(B, 8, 3)`` boxes.Examples:>>> input = torch.arange(16, dtype=torch.float32).reshape((1, 1, 4, 4))@@ -365,7 +446,16 @@ def crop_by_transform_mat(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- """Perform crop transform on 2D images (4D torch.Tensor) given a perspective transformation matrix.+ """Crop images by a sourceādestination pixel matrix, affine or homography, into a given output size.++ ``patches = crop_by_transform_mat(input_tensor, transform, out_size)`` takes images ``(B, C, H, W)`` and a+ sourceādestination pixel matrix per image, affine ``(B, 2, 3)`` or homography ``(B, 3, 3)`` (a batch of 1 serves+ every image), and returns the ``(B, C, h, w)`` destination window starting at pixel ``(0, 0)`` for+ ``out_size = (h, w)``. It calls :func:`~kornia.geometry.transform.warp_affine` for an affine matrix and+ :func:`~kornia.geometry.transform.warp_perspective` for a homography, after casting the matrix to the image dtype+ and expanding it to the batch: a float64 or integer matrix crops a float32 image, which those two reject, and a+ batch-1 homography serves a batch, which ``warp_perspective`` alone does not allow. To build the matrix from boxes,+ use :func:`crop_by_boxes` or :func:`crop_and_resize`; for volumes, :func:`crop_by_transform_mat3d`.Convention:- input: ``(B, C, H, W)``; ``out_size`` is ``(h, w)``@@ -377,9 +467,9 @@ def crop_by_transform_mat(``out_size`` dimension of ``1`` is handled like any other sizeArgs:- input_tensor: the 2D image torch.Tensor with shape (B, C, H, W).- transform: a perspective transformation matrix with shape (B, 3, 3), or an- affine matrix with shape (B, 2, 3) that takes the cheaper warp_affine path.+ input_tensor: the 2D image torch.Tensor with shape ``(B, C, H, W)``.+ transform: a perspective transformation matrix with shape ``(B, 3, 3)``, or an+ affine matrix with shape ``(B, 2, 3)`` that takes the cheaper warp_affine path.out_size: size of the output image (height, width).mode: interpolation mode to calculate output values``'bilinear'`` | ``'nearest'``.@@ -388,7 +478,13 @@ def crop_by_transform_mat(align_corners: mode for grid_generation.Returns:- the output torch.Tensor with patches.+ the patches, with shape ``(B, C, h, w)``.++ See Also:+ - :func:`warp_affine`: the affine warp it calls, with a ``fill_value`` for the padding.+ - :func:`warp_perspective`: the homography warp it calls, with a ``fill_value`` for the padding.+ - :func:`crop_by_boxes`: builds the homography from a source and a destination box.+ - :func:`crop_by_transform_mat3d`: the same for volumes, by ``(B, 3, 4)`` or ``(B, 4, 4)`` matrices."""# simulate broadcasting@@ -458,11 +554,21 @@ def crop_by_indices(antialias: bool = False,shape_compensation: str = "resize",) -> torch.Tensor:- """Crop tensors with naive indices.+ """Crop axis-aligned boxes by integer slicing, then resize or pad each crop to a common size.++ ``patches = crop_by_indices(input_tensor, src_box, size)`` takes images ``(B, C, H, W)`` and one box per image,+ ``(B, 4, 2)``, slices each box out with integer indices and returns ``(B, C, h, w)`` for ``size = (h, w)``. Outside+ graph export (see the notes) it slices by the top-left corner, the right edge (``x`` of vertex 1) and the bottom+ edge (``y`` of vertex 3) of a box, and truncates float coordinates toward zero; without ``size`` it takes the size+ from the box as :func:`~kornia.geometry.bbox.infer_bbox_shape` does, whose height reads ``y`` of vertex 2, and every+ box must have the same size. A slice of another size than ``size``, from a box of another size or one clipped by the+ image border, is resized (``shape_compensation='resize'``) or zero-padded or trimmed to it (``'pad'``).+ The crop is differentiable with respect to the image, not the box: for sub-pixel, rotated or differentiable boxes,+ use :func:`crop_and_resize`.Convention:- input: ``(B, C, H, W)``; ``size`` is ``(h, w)`` if given, else inferred- from ``src_box``+ from ``src_box``; a single box does not broadcast over a batch of images- ``src_box``: ``(B, 4, 2)`` corner points in ``(x, y)`` order top-left,top-right, bottom-right, bottom-left ā same convention as :func:`crop_and_resize`- unlike the other crop operators in this module: ``interpolation=`` (not@@ -484,9 +590,9 @@ def crop_by_indices(back to Python and breaks the graph. Graph export is described in the note below.Args:- input_tensor: the 2D image torch.Tensor with shape (B, C, H, W).- src_box: a torch.Tensor with shape (B, 4, 2) containing the coordinates of the bounding boxes- to be extracted. The torch.Tensor must have the shape of Bx4x2, where each box is defined+ input_tensor: the 2D image torch.Tensor with shape ``(B, C, H, W)``.+ src_box: a torch.Tensor with shape ``(B, 4, 2)`` containing the coordinates of the bounding boxes+ to be extracted, where each box is definedin the clockwise order: top-left, top-right, bottom-right and bottom-left.The coordinates must be in x, y order.size: output size. An auto resize or F.pad will be performed according to ``shape_compensation``@@ -501,10 +607,20 @@ def crop_by_indices(or resized.Returns:- The cropped torch.Tensor with shape :math:`(B, C, h, w)`, where- :math:`(h, w)` is ``size`` if given, otherwise the shape inferred+ The cropped torch.Tensor with shape ``(B, C, h, w)``, where+ ``(h, w)`` is ``size`` if given, otherwise the shape inferredfrom ``src_box``.+ Raises:+ ValueError: if ``size`` is ``None`` and the boxes differ in size.++ See Also:+ - :func:`crop_and_resize`: resamples any four-corner box, differentiably with respect to its corners.+ - :class:`CenterCrop2D`: slices the central window with this function in its default mode, unless the size+ exceeds the input on an axis, where it zero-pads and slices directly.+ - :class:`~kornia.augmentation.RandomCrop`: random boxes, sliced with this function in its default mode, in an+ augmentation pipeline.+.. note::Under graph export (:func:`torch.export.export` / :func:`torch.onnx.export`) the boxcoordinates cannot be read back to Python, so the crop is captured as the same warp as@@ -674,7 +790,16 @@ def _crop_by_indices_export(class CenterCrop2D(nn.Module):- """Center crop the input torch.Tensor.+ """Crop the central window of each image to a given size, by slicing or resampling (module form).++ Build ``cropper = CenterCrop2D(size)`` with an ``int`` or an ``(h, w)`` tuple, then ``out = cropper(input)`` returns+ the central ``(B, C, h, w)`` windows of ``(B, C, H, W)`` images, at the offsets of :func:`center_crop`. The default+ ``cropping_mode='slice'`` slices with integer indices, through :func:`crop_by_indices`, or after zero-padding when+ ``size`` exceeds the input on an axis; ``'resample'`` samples the same windows, up to float rounding, by a+ translation through :func:`~kornia.geometry.transform.warp_affine` with the ``resample`` mode, except that with+ ``align_corners=True`` it repeats a one-pixel input axis that ``'slice'`` zero-pads+ (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). For one call, use+ :func:`center_crop`; in an augmentation pipeline, :class:`~kornia.augmentation.CenterCrop`.Convention:- ``align_corners`` (and ``resample``) only take effect when@@ -683,13 +808,22 @@ class CenterCrop2D(nn.Module):- See the convention block of :func:`~kornia.geometry.transform.center_crop`.Args:- size: Size (h, w) in pixels of the resized region or just one side.+ size: Size ``(h, w)`` in pixels of the cropped region, or one ``int`` for both sides; a list raises+ ``TypeError``.align_corners: interpolation flag.resample: Resampling mode.cropping_mode: Cropping mode, "resample" or "slice".Note:- For JIT, the cropping mode must be "resample".+ ``torch.jit.script`` compiles neither cropping mode, and ``torch.jit.trace`` records the crop offsets of the+ traced input size.++ See Also:+ - :func:`center_crop`: the one-call form, which always resamples.+ - :func:`crop_by_indices`: the integer slicing of the default mode, for any axis-aligned box.+ - :class:`~kornia.augmentation.CenterCrop`: the same windows as an augmentation, for augmentation pipelines;+ it raises for a ``size`` larger than the input instead of padding.+"""def __init__(@@ -732,7 +866,7 @@ class CenterCrop2D(nn.Module):source height, and :math:`W` is source width.Returns:- Center-cropped tensor with shape :math:`(B, C, h, w)`, where+ Center-cropped tensor with shape ``(B, C, h, w)``, where:math:`h` and :math:`w` are the requested crop height and widthstored in ``self.size``."""diff --git a/kornia/geometry/transform/crop3d.py b/kornia/geometry/transform/crop3d.pyindex 3a0a865dd..b4ab50bd7 100644--- a/kornia/geometry/transform/crop3d.py+++ b/kornia/geometry/transform/crop3d.py@@ -15,6 +15,31 @@# limitations under the License.#+"""Crop volume regions by boxes or by a matrix, resampled to a given output size.++The crops take channels-first volumes ``(B, C, D, H, W)``. A box is ``(B, 8, 3)``: eight ``(x, y, z)`` voxel+corners, the front face (smaller ``z``) and then the back face, each face top-left, top-right, bottom-right,+bottom-left, with **inclusive** coordinates as in the 2D crops of+:func:`~kornia.geometry.transform.crop_and_resize`. Sizes are ``(d, h, w)``. Every crop resamples by a+sourceādestination voxel matrix through :func:`~kornia.geometry.transform.warp_affine3d`, into an output window that+starts at voxel ``(0, 0, 0)``. Pass ``align_corners=True``: at ``False`` the matrix is normalized for corner-aligned+voxels while ``grid_sample`` reads half-voxel ones, so the samples miss the box, and even a box of the output size is+interpolated rather than copied (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). :func:`crop_and_resize3d`+and :func:`crop_by_boxes3d` default to ``False``. In eager mode the box crops reject, with ``AssertionError``, a box+whose four edges along an axis span different extents on it, or one with a non-finite coordinate. They raise+``torch.linalg.LinAlgError`` for a box or an output size with a one-voxel axis+(`#4747 <https://github.com/kornia/kornia/issues/4747>`_).++Which function to use:++- A box resampled to a fixed ``(d, h, w)``: :func:`crop_and_resize3d`; mapped onto a second box whose size sets the+ output: :func:`crop_by_boxes3d`.+- A crop given as a sourceādestination voxel matrix, ``(B, 3, 4)`` or ``(B, 4, 4)``: :func:`crop_by_transform_mat3d`.+- The central window of each volume: :func:`center_crop3d`.+- For images, use :func:`~kornia.geometry.transform.crop_and_resize` and the other 2D crops; in an augmentation+ pipeline, :class:`~kornia.augmentation.RandomCrop3D` or :class:`~kornia.augmentation.CenterCrop3D`.+"""+from typing import Tupleimport torch@@ -33,7 +58,15 @@ def crop_and_resize3d(interpolation: str = "bilinear",align_corners: bool = False,) -> torch.Tensor:- r"""Extract crops from 3D volumes (5D tensor) and resize them.+ r"""Crop an eight-corner box from each volume and resample it to a fixed output size.++ ``patches = crop_and_resize3d(tensor, boxes, size)`` takes volumes ``(B, C, D, H, W)`` and boxes ``(B, 8, 3)``, one+ per volume or ``(1, 8, 3)`` for all of them, and returns the ``(B, C, d, h, w)`` patches for ``size = (d, h, w)``.+ With ``align_corners=True`` the corners of a parallelepiped box land on the corner voxels of the patch; the warp+ keeps only the top three rows of the 3D homography between the two, so the corners of any other box miss+ (`#6436 <https://github.com/kornia/kornia/issues/6436>`_). Pass ``align_corners=True``, since the default+ ``False`` samples off the box (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For images, use+ :func:`crop_and_resize`; to place the crop by a second box, :func:`crop_by_boxes3d`.Convention:- input: ``(B, C, D, H, W)``; ``size`` is ``(d, h, w)``@@ -44,11 +77,13 @@ def crop_and_resize3d(reproducing the exact integer-voxel slice requires ``align_corners=True`` āthe default ``False`` interpolates instead- align_corners: ``False`` by default+ - a single volume does not broadcast over a batch of boxes; a box or ``size`` with a one-voxel axis raises+ ``torch.linalg.LinAlgError`` (`#4747 <https://github.com/kornia/kornia/issues/4747>`_)Args:- tensor: the 3D volume tensor with shape (B, C, D, H, W).- boxes: a tensor with shape (B, 8, 3) containing the coordinates of the bounding boxes- to be extracted. The tensor must have the shape of Bx8x3, where each box is defined in the clockwise+ tensor: the 3D volume tensor with shape ``(B, C, D, H, W)``.+ boxes: a tensor with shape ``(B, 8, 3)`` containing the coordinates of the bounding boxes+ to be extracted, where each box is defined in the clockwiseorder: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left,back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in x, y, z order.size: a tuple with the depth, height and width that will be@@ -57,7 +92,18 @@ def crop_and_resize3d(align_corners: mode for grid_generation.Returns:- tensor containing the patches with shape (Bx)CxN1xN2xN3.+ tensor containing the patches with shape ``(B, C, d, h, w)``.++ Raises:+ AssertionError: if a box has a non-finite coordinate, or edges along one axis that span different extents on+ it.++ See Also:+ - :func:`crop_and_resize`: the same for images and ``(B, 4, 2)`` boxes, where, along an axis longer than one+ pixel, ``align_corners`` changes only ``'reflection'`` padding and, with ``mode='nearest'``, which pixel a+ sample exactly halfway takes.+ - :func:`crop_by_boxes3d`: maps the box onto a second box, whose size sets the output size.+ - :func:`center_crop3d`: the central window of each volume, with ``align_corners=True`` by default.Example:>>> input = torch.arange(64, dtype=torch.float32).view(1, 1, 4, 4, 4)@@ -141,7 +187,14 @@ def crop_and_resize3d(def center_crop3d(tensor: torch.Tensor, size: Tuple[int, int, int], interpolation: str = "bilinear", align_corners: bool = True) -> torch.Tensor:- r"""Crop the 3D volumes (5D tensor) at the center.+ r"""Crop the central window of each volume to a given size.++ ``patches = center_crop3d(tensor, size)`` takes volumes ``(B, C, D, H, W)`` and returns their central+ ``(B, C, d, h, w)`` windows for ``size = (d, h, w)``: with the default ``align_corners=True``, a sub-volume of the+ input, with the extra voxel of an odd margin at the back/bottom/right. A ``size`` larger than the input pads with+ zeros, except along a one-voxel input axis, which ``align_corners=True`` repeats+ (`#6433 <https://github.com/kornia/kornia/issues/6433>`_). For images, use :func:`center_crop`; for a crop at any+ box, :func:`crop_and_resize3d`.Convention:- input: ``(B, C, D, H, W)``; ``size`` is ``(d, h, w)``@@ -150,17 +203,25 @@ def center_crop3d(as in :class:`~kornia.augmentation.CenterCrop3D`, so with the default ``align_corners=True`` the output is asub-volume of the input. With an odd margin the extra voxel is left at the back/bottom/right.``align_corners=False`` resamples off the voxel grid even for an even margin (see- :func:`crop_and_resize3d`).+ :func:`crop_and_resize3d`, `#4503 <https://github.com/kornia/kornia/issues/4503>`_).+ - a ``size`` with a one-voxel axis raises ``torch.linalg.LinAlgError``+ (`#4747 <https://github.com/kornia/kornia/issues/4747>`_)Args:- tensor: the 3D volume tensor with shape (B, C, D, H, W).+ tensor: the 3D volume tensor with shape ``(B, C, D, H, W)``.size: a tuple with the expected depth, height and widthof the output patch.interpolation: Interpolation flag.align_corners : mode for grid_generation.Returns:- the output tensor with patches.+ the central windows, with shape ``(B, C, d, h, w)``.++ See Also:+ - :func:`center_crop`: the central window of each image.+ - :func:`crop_and_resize3d`: a window at any box, resampled to a fixed size.+ - :class:`~kornia.augmentation.CenterCrop3D`: the same windows as an augmentation, for augmentation pipelines;+ it raises for a ``size`` larger than the input instead of padding.Examples:>>> input = torch.arange(64, dtype=torch.float32).view(1, 1, 4, 4, 4)@@ -276,12 +337,18 @@ def crop_by_boxes3d(interpolation: str = "bilinear",align_corners: bool = False,) -> torch.Tensor:- r"""Perform crop transform on 3D volumes (5D tensor) by bounding boxes.-- Given an input tensor, this function selected the interested areas by the provided bounding boxes (src_box).- Then the selected areas would be fitted into the targeted bounding boxes (dst_box) by a perspective transformation.- So far, the ragged tensor is not supported by PyTorch right now. This function hereby requires the bounding boxes- in a batch must be rectangles with same width, height and depth.+ r"""Crop volumes by mapping a source box onto a destination box, whose size sets the output size.++ ``patches = crop_by_boxes3d(tensor, src_box, dst_box)`` takes volumes ``(B, C, D, H, W)`` and two boxes per+ volume, ``(B, 8, 3)`` each (both ``(1, 8, 3)`` for all volumes), warps each volume by the top three rows of the+ 3D homography computed from the corners of ``src_box`` and ``dst_box``, which takes every corner onto its+ destination only when an affine map relates the two boxes, as it relates any two parallelepipeds+ (`#6436 <https://github.com/kornia/kornia/issues/6436>`_), and returns the ``(B, C, d, h, w)`` window that+ starts at voxel ``(0, 0, 0)``, where+ ``(d, h, w)`` is the inclusive size of ``dst_box``; every ``dst_box`` in the batch must have that size. Pass+ ``align_corners=True``: the default ``False`` samples off the box+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). When the output size is known, :func:`crop_and_resize3d`+ takes it instead of a second box; for images, use :func:`crop_by_boxes`.Convention:- input: ``(B, C, D, H, W)``@@ -292,20 +359,29 @@ def crop_by_boxes3d(- align_corners: ``False`` by defaultArgs:- tensor : the 3D volume tensor with shape (B, C, D, H, W).- src_box : a tensor with shape (B, 8, 3) containing the coordinates of the bounding boxes- to be extracted. The tensor must have the shape of Bx8x3, where each box is defined in the clockwise+ tensor : the 3D volume tensor with shape ``(B, C, D, H, W)``.+ src_box : a tensor with shape ``(B, 8, 3)`` containing the coordinates of the bounding boxes+ to be extracted, where each box is defined in the clockwiseorder: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left,back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in x, y, z order.- dst_box: a tensor with shape (B, 8, 3) containing the coordinates of the bounding boxes- to be placed. The tensor must have the shape of Bx8x3, where each box is defined in the clockwise+ dst_box: a tensor with shape ``(B, 8, 3)`` containing the coordinates of the bounding boxes+ to be placed, where each box is defined in the clockwiseorder: front-top-left, front-top-right, front-bottom-right, front-bottom-left, back-top-left,back-top-right, back-bottom-right, back-bottom-left. The coordinates must be in x, y, z order.interpolation: Interpolation flag.align_corners: mode for grid_generation.Returns:- the output tensor with patches.+ the patches, with shape ``(B, C, d, h, w)`` for the size ``(d, h, w)`` of ``dst_box``.++ Raises:+ AssertionError: if the boxes of ``dst_box`` differ in size, or a box has a non-finite coordinate or edges along+ one axis that span different extents on it.++ See Also:+ - :func:`crop_by_boxes`: the same for images and ``(B, 4, 2)`` boxes.+ - :func:`crop_and_resize3d`: takes the output size as ``size`` and places the box at the patch corners.+ - :func:`crop_by_transform_mat3d`: takes the sourceādestination matrix itself.Examples:>>> input = torch.tensor([[[@@ -402,7 +478,15 @@ def crop_by_transform_mat3d(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- """Perform crop transform on 3D volumes (5D tensor) given a perspective transformation matrix.+ """Crop volumes by a sourceādestination voxel matrix into a given output size.++ ``patches = crop_by_transform_mat3d(tensor, transform, out_size)`` takes volumes ``(B, C, D, H, W)`` and a+ sourceādestination voxel matrix per volume, ``(B, 3, 4)`` or ``(B, 4, 4)`` (a batch of 1 serves every volume, which+ :func:`warp_affine3d` alone does not allow), and returns the ``(B, C, d, h, w)`` window that starts at voxel+ ``(0, 0, 0)`` for ``out_size = (d, h, w)``. It warps with :func:`warp_affine3d` and uses only the top three rows,+ so a projective last row is ignored. Unlike :func:`crop_by_transform_mat`, it does not cast the matrix: its dtype+ must be the volume's. For images, use :func:`crop_by_transform_mat`; to build the matrix from boxes,+ :func:`crop_by_boxes3d`.Convention:- input: ``(B, C, D, H, W)``; ``out_size`` is ``(d, h, w)``@@ -410,12 +494,13 @@ def crop_by_transform_mat3d(either ``(B, 3, 4)`` affine or ``(B, 4, 4)`` homogeneous; only the topthree rows are used, so a projective last row is ignored (unlike the 2D:func:`crop_by_transform_mat`)- - align_corners: ``True`` by default+ - align_corners: ``True`` by default; ``False`` warps by a matrix normalized for corner-aligned voxels+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_)Args:- tensor: the 3D volume tensor with shape (B, C, D, H, W).+ tensor: the 3D volume tensor with shape ``(B, C, D, H, W)``.transform: the source->destination pixel transform, either affine with- shape (B, 3, 4) or homogeneous with shape (B, 4, 4).+ shape ``(B, 3, 4)`` or homogeneous with shape ``(B, 4, 4)``.out_size: size of the output image (depth, height, width).mode: interpolation mode to calculate output values``'bilinear'`` | ``'nearest'``.@@ -424,7 +509,12 @@ def crop_by_transform_mat3d(align_corners: mode for grid_generation.Returns:- the output tensor with patches.+ the patches, with shape ``(B, C, d, h, w)``.++ See Also:+ - :func:`crop_by_transform_mat`: the same for images, which also applies a projective row.+ - :func:`warp_affine3d`: the warp it calls, which takes one ``(B, 3, 4)`` matrix per volume.+ - :func:`crop_by_boxes3d`: builds the matrix from a source and a destination box."""# simulate broadcastingdiff --git a/kornia/geometry/transform/elastic_transform.py b/kornia/geometry/transform/elastic_transform.pyindex 3b5d35a2d..008beaf9f 100644--- a/kornia/geometry/transform/elastic_transform.py+++ b/kornia/geometry/transform/elastic_transform.py@@ -15,6 +15,16 @@# limitations under the License.#+"""Deform images elastically by a smoothed displacement field, as in Simard et al.++Which function to use:++- A displacement field from your own noise, smoothed by a Gaussian and scaled per axis: :func:`elastic_transform2d`.+- For random elastic deformations in an augmentation pipeline, use :class:`~kornia.augmentation.RandomElasticTransform`;+ for a smooth deformation set by control points, :func:`~kornia.geometry.transform.warp_image_tps`; to sample an+ image at per-pixel coordinates of your own, :func:`~kornia.geometry.transform.remap`.+"""+from typing import Tuple, Unionimport torch@@ -39,7 +49,16 @@ def elastic_transform2d(mode: str = "bilinear",padding_mode: str = "zeros",) -> torch.Tensor:- r"""Apply elastic transform of images as described in :cite:`Simard2003BestPF`.+ r"""Deform images by a displacement field smoothed from noise, as described in :cite:`Simard2003BestPF`.++ ``out = elastic_transform2d(image, noise)`` takes images ``(B, C, H, W)`` and a noise field ``(B, 2, H, W)``,+ smooths each channel of the noise with a Gaussian of ``kernel_size`` and ``sigma``, scales it by ``alpha`` and+ returns the ``(B, C, H, W)`` images sampled at the displaced pixels. The displacements are in the normalized+ :math:`[-1, 1]` coordinates of ``grid_sample``: an x displacement of 1 moves a sample by :math:`W / 2` pixels at the+ default ``align_corners=False`` and by :math:`(W - 1) / 2` at ``True``, and a y displacement likewise by the+ height. One Gaussian and one ``alpha`` serve the whole batch. For random fields in an augmentation pipeline, use+ :class:`~kornia.augmentation.RandomElasticTransform`; for a smooth warp set by control points,+ :func:`~kornia.geometry.transform.warp_image_tps`... image:: _static/img/elastic_transform2d.png@@ -48,7 +67,8 @@ def elastic_transform2d(channel 0 the x- and channel 1 the y-displacement of the samplinggrid (positive x-noise shifts image content left)- displacements are smoothed with a Gaussian (``kernel_size``,- ``sigma``) and scaled by ``alpha``+ ``sigma``) and scaled by ``alpha``; ``kernel_size`` and ``sigma`` are ``(y, x)`` pairs, but ``alpha`` is+ ``(x, y)``- align_corners: ``False`` by default (the matrix warps default to ``True``); zero``noise`` is an identity under either setting- coordinates displaced outside the image are filled by ``padding_mode`` as in ``grid_sample``:@@ -59,17 +79,23 @@ def elastic_transform2d(noise: Noise image used to spatially transform the input image. Sameresolution as the input image with shape ``(B, 2, H, W)``. The coordinates orderit is expected to be in x-y.- kernel_size: the size of the Gaussian kernel, in the y and x directions, respectively.+ kernel_size: the size of the Gaussian kernel, in the y and x directions, respectively; both odd.sigma: The standard deviation of the Gaussian in the y and x directions,- respectively. Larger sigma results in smaller pixel displacements.+ respectively, as a pair or a ``(2,)`` tensor. Larger sigma results in smaller pixel displacements.alpha : The scaling factor that controls the intensity of the deformation- in the x and y directions, respectively.+ in the x and y directions, respectively, as a pair or a ``(2,)`` tensor.align_corners: Interpolation flag used by ``grid_sample``.mode: Interpolation mode used by ``grid_sample``. Either ``'bilinear'`` or ``'nearest'``.padding_mode: The padding used by ``grid_sample``. Either ``'zeros'``, ``'border'`` or ``'reflection'``.Returns:- the elastically transformed input image with shape ``(B,C,H,W)``.+ the elastically transformed input image with shape ``(B, C, H, W)``.++ See Also:+ - :class:`~kornia.augmentation.RandomElasticTransform`: draws uniform noise in :math:`[-1, 1]` for each call and+ applies it with probability ``p``, in an augmentation pipeline.+ - :func:`~kornia.geometry.transform.warp_image_tps`: a smooth deformation set by control-point pairs.+ - :func:`~kornia.geometry.transform.remap`: samples an image at per-pixel coordinates given directly.Example:>>> image = torch.rand(1, 3, 5, 5)diff --git a/kornia/geometry/transform/flips.py b/kornia/geometry/transform/flips.pyindex c74bef934..f473b281a 100644--- a/kornia/geometry/transform/flips.py+++ b/kornia/geometry/transform/flips.py@@ -15,6 +15,20 @@# limitations under the License.#+"""Mirror images left to right or top to bottom, or turn them by 180 degrees, by reversing tensor axes.++The flips reverse the last axes of a tensor of any batch shape: :func:`hflip` the last one (the width ``W``),+:func:`vflip` the second to last (the height ``H``) and :func:`rot180` both. They copy elements without interpolation,+so they are exact for every dtype and differentiable, and their module forms take no arguments.++Which function to use:++- Mirror left to right: :func:`hflip` or :class:`Hflip`; top to bottom: :func:`vflip` or :class:`Vflip`.+- Turn by a half turn: :func:`rot180` or :class:`Rot180`.+- To rotate by any angle, use :func:`~kornia.geometry.transform.rotate`; for random flips in an augmentation+ pipeline, :class:`~kornia.augmentation.RandomHorizontalFlip` or :class:`~kornia.augmentation.RandomVerticalFlip`.+"""+import torchfrom torch import nn@@ -22,16 +36,22 @@ __all__ = ["Hflip", "Rot180", "Vflip", "hflip", "rot180", "vflip"]class Vflip(nn.Module):- r"""Vertically flip a torch.Tensor image or a batch of torch.Tensor images.+ r"""Flip images top to bottom by reversing the height axis (module form).++ Build ``flipper = Vflip()``, which takes no arguments, then ``out = flipper(input)`` reverses the rows of+ ``(*, H, W)`` tensors as :func:`vflip` does. To mirror left to right, use :class:`Hflip`; for a half turn,+ :class:`Rot180`.Convention:- See the convention block of :func:`~kornia.geometry.transform.vflip`.- Args:- input: input torch.Tensor.-Returns:- The vertically flipped image torch.Tensor.+ The vertically flipped image torch.Tensor, with the shape of the input.++ See Also:+ - :func:`vflip`: the one-call form.+ - :class:`Hflip`: mirrors left to right instead.+ - :class:`~kornia.augmentation.RandomVerticalFlip`: flips with a probability, in an augmentation pipeline.Examples:>>> vflip = Vflip()@@ -66,16 +86,22 @@ class Vflip(nn.Module):class Hflip(nn.Module):- r"""Horizontally flip a torch.Tensor image or a batch of torch.Tensor images.+ r"""Mirror images left to right by reversing the width axis (module form).++ Build ``flipper = Hflip()``, which takes no arguments, then ``out = flipper(input)`` reverses the columns of+ ``(*, W)`` tensors as :func:`hflip` does. To flip top to bottom, use :class:`Vflip`; for a half turn,+ :class:`Rot180`.Convention:- See the convention block of :func:`~kornia.geometry.transform.hflip`.- Args:- input: input torch.Tensor.-Returns:- The horizontally flipped image torch.Tensor.+ The horizontally flipped image torch.Tensor, with the shape of the input.++ See Also:+ - :func:`hflip`: the one-call form.+ - :class:`Vflip`: flips top to bottom instead.+ - :class:`~kornia.augmentation.RandomHorizontalFlip`: flips with a probability, in an augmentation pipeline.Examples:>>> hflip = Hflip()@@ -110,13 +136,21 @@ class Hflip(nn.Module):class Rot180(nn.Module):- r"""Rotate a torch.Tensor image or a batch of torch.Tensor images 180 degrees.+ r"""Turn images by 180 degrees by reversing the height and width axes (module form).++ Build ``rotator = Rot180()``, which takes no arguments, then ``out = rotator(input)`` reverses both the rows and the+ columns of ``(*, H, W)`` tensors as :func:`rot180` does, without interpolation. To rotate by a stored angle, use+ :class:`Rotate`.Convention:- See the convention block of :func:`~kornia.geometry.transform.rot180`.- Args:- input: input torch.Tensor.+ Returns:+ The rotated image torch.Tensor, with the shape of the input.++ See Also:+ - :func:`rot180`: the one-call form.+ - :class:`Rotate`: rotates by any stored angle, with interpolation.Examples:>>> rot180 = Rot180()@@ -151,7 +185,11 @@ class Rot180(nn.Module):def rot180(input: torch.Tensor) -> torch.Tensor:- r"""Rotate a torch.Tensor image or a batch of torch.Tensor images 180 degrees.+ r"""Turn images by 180 degrees by reversing the height and width axes.++ ``out = rot180(input)`` reverses the last two axes of ``(*, H, W)`` tensors and returns the same shape: the image+ turned by a half turn, exactly, without the interpolation of :func:`rotate`. For any other angle, or a turn about+ another centre, use :func:`rotate`; for the module form, :class:`Rot180`... image:: _static/img/rot180.png@@ -165,14 +203,25 @@ def rot180(input: torch.Tensor) -> torch.Tensor:input: input torch.Tensor.Returns:- The rotated image torch.Tensor.+ The rotated image torch.Tensor, with the shape of the input.++ See Also:+ - :class:`Rot180`: the module form.+ - :func:`rotate`: rotates by any angle about the centre or a given point, with interpolation; at 180 degrees+ about the default centre it matches ``rot180`` up to float rounding.+ - :func:`hflip`: reverses the width axis only.+ - :func:`vflip`: reverses the height axis only."""return torch.flip(input, [-2, -1])def hflip(input: torch.Tensor) -> torch.Tensor:- r"""Horizontally flip a torch.Tensor image or a batch of torch.Tensor images.+ r"""Mirror images left to right by reversing the width axis.++ ``out = hflip(input)`` reverses the last axis of ``(*, W)`` tensors, the width of ``(*, H, W)`` images, and returns+ the same shape. To flip top to bottom, use :func:`vflip`; for a half turn, :func:`rot180`; for random flips in an+ augmentation pipeline, :class:`~kornia.augmentation.RandomHorizontalFlip`... image:: _static/img/hflip.png@@ -184,14 +233,24 @@ def hflip(input: torch.Tensor) -> torch.Tensor:input: input torch.Tensor.Returns:- The horizontally flipped image torch.Tensor.+ The horizontally flipped image torch.Tensor, with the shape of the input.++ See Also:+ - :class:`Hflip`: the module form.+ - :func:`vflip`: reverses the height axis instead.+ - :func:`rot180`: reverses both axes.+ - :class:`~kornia.augmentation.RandomHorizontalFlip`: flips with a probability, in an augmentation pipeline."""return input.flip(-1).contiguous()def vflip(input: torch.Tensor) -> torch.Tensor:- r"""Vertically flip a torch.Tensor image or a batch of torch.Tensor images.+ r"""Flip images top to bottom by reversing the height axis.++ ``out = vflip(input)`` reverses the second-to-last axis of ``(*, H, W)`` tensors and returns the same shape. To+ mirror left to right, use :func:`hflip`; for a half turn, :func:`rot180`; for random flips in an augmentation+ pipeline, :class:`~kornia.augmentation.RandomVerticalFlip`... image:: _static/img/vflip.png@@ -204,7 +263,13 @@ def vflip(input: torch.Tensor) -> torch.Tensor:input: input torch.Tensor.Returns:- The vertically flipped image torch.Tensor.+ The vertically flipped image torch.Tensor, with the shape of the input.++ See Also:+ - :class:`Vflip`: the module form.+ - :func:`hflip`: reverses the width axis instead.+ - :func:`rot180`: reverses both axes.+ - :class:`~kornia.augmentation.RandomVerticalFlip`: flips with a probability, in an augmentation pipeline."""return input.flip(-2).contiguous()diff --git a/kornia/geometry/transform/homography_warper.py b/kornia/geometry/transform/homography_warper.pyindex 88c601e5c..d5ac85aca 100644--- a/kornia/geometry/transform/homography_warper.py+++ b/kornia/geometry/transform/homography_warper.py@@ -15,6 +15,21 @@# limitations under the License.#+"""Warp images by homographies in module form, optionally reusing a precomputed sampling grid.++A warper reads ``src_homo_dst`` as the destinationāsource homography; :class:`HomographyWarper` takes it in+normalized :math:`[-1, 1]` coordinates by default, as :func:`~kornia.geometry.transform.homography_warp` does.++Which class to use:++- Warp ``(N, C, H, W)`` images to a fixed ``(height, width)``, optionally reusing one precomputed grid:+ :class:`HomographyWarper`.+- Write another warper for :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator`: subclass+ :class:`BaseWarper`.+- For a sourceādestination homography in pixel coordinates, use+ :func:`~kornia.geometry.transform.warp_perspective` instead.+"""+from __future__ import annotationsfrom abc import abstractmethod@@ -32,11 +47,24 @@ __all__ = ["BaseWarper", "HomographyWarper"]class BaseWarper(nn.Module):- """Provide a base class for homography-based image warping.+ """Declare the interface of a homography warper module that ImageRegistrator can call.++ ``BaseWarper(height, width)`` only stores the destination ``height`` and ``width``. A subclass implements+ ``warper(patch_src, src_homo_dst)``, which samples a ``(B, C, H, W)`` source on that destination grid by a+ ``(B, 3, 3)`` destinationāsource homography, and ``precompute_warp_grid(src_homo_dst)``, which caches the grid.+ :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator` takes such a class as ``warper`` and calls+ it as ``warper(height, width)``. The two methods are marked abstract, but :class:`torch.nn.Module` does not enforce+ it: ``BaseWarper`` itself constructs, and its ``forward`` returns ``None``+ (`#6399 <https://github.com/kornia/kornia/issues/6399>`_). To warp images, use :class:`HomographyWarper`.Convention:- subclasses receive ``src_homo_dst`` as the destinationāsource homography+ See Also:+ - :class:`HomographyWarper`: the implementation kornia ships, in normalized or pixel coordinates.+ - :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator`: registers two images by+ optimizing a transform, and warps with a warper class of this interface.+"""def __init__(self, height: int, width: int, *args: Any, **kwargs: Any) -> None:@@ -75,10 +103,15 @@ class BaseWarper(nn.Module):class HomographyWarper(BaseWarper):- r"""Warp tensors by a normalized destinationāsource homography (default ``normalized_coordinates=True``).-- For a sourceādestination homography in pixel coordinates, as :class:`~kornia.geometry.ransac.RANSAC` and- :func:`~kornia.geometry.homography.find_homography_dlt` return it, use+ r"""Warp images to a fixed size by a normalized destinationāsource homography (module form).++ Build ``warper = HomographyWarper(height, width)``, then ``out = warper(patch_src, src_homo_dst)`` warps+ ``(N, C, H, W)`` images by ``(N, 3, 3)`` destinationāsource homographies in normalized :math:`[-1, 1]`+ coordinates and returns ``(N, C, height, width)``, as :func:`homography_warp` with ``dsize=(height, width)`` does.+ To reuse one homography, call ``warper.precompute_warp_grid(src_homo_dst)`` once and then ``warper(patch_src)``; a+ homography passed to the call takes precedence over the cached grid, and a call with neither raises+ ``RuntimeError``. For a sourceādestination homography in pixel coordinates, as+ :class:`~kornia.geometry.ransac.RANSAC` and :func:`~kornia.geometry.homography.find_homography_dlt` return it, use:func:`~kornia.geometry.transform.warp_perspective`. This class has no ``normalized_homography`` switch:``normalized_coordinates=False`` still reads a destinationāsource homography, in pixels.@@ -91,7 +124,7 @@ class HomographyWarper(BaseWarper):coordinates and sampling conventions.- See the convention block of :func:`homography_warp`.- - with ``normalized_coordinates=False`` a grid from :meth:`precompute_warp_grid` holds source pixel+ - with ``normalized_coordinates=False`` a grid from ``precompute_warp_grid`` holds source pixelcoordinates computed as in :func:`homography_warp` (in at least float32, unaffected by autocast), normalizedat sampling time with the size of the patch passed to ``forward`` and then cast to the homography's dtype@@ -105,6 +138,15 @@ class HomographyWarper(BaseWarper):(``True``) or in pixel coordinates (``False``), as in :func:`homography_warp`.align_corners: interpolation flag.+ See Also:+ - :func:`homography_warp`: the one-call form, with a ``normalized_homography`` switch for pixel+ sourceādestination homographies.+ - :func:`warp_perspective`: takes the sourceādestination homography in pixel coordinates.+ - :class:`BaseWarper`: the interface this class implements, for+ :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator`.+ - :class:`~kornia.geometry.transform.image_registrator.ImageRegistrator`: fits a homography between two+ images, warping with this class by default.+"""_warped_grid: Optional[torch.Tensor]diff --git a/kornia/geometry/transform/image_registrator.py b/kornia/geometry/transform/image_registrator.pyindex 627fe53e8..025b16134 100644--- a/kornia/geometry/transform/image_registrator.py+++ b/kornia/geometry/transform/image_registrator.py@@ -15,6 +15,25 @@# limitations under the License.#+"""Register two images by optimizing a homography or a similarity by gradient descent, coarse to fine.++:class:`ImageRegistrator` takes a pair of ``(1, C, H, W)`` images, builds a Gaussian pyramid of each with+:func:`~kornia.geometry.transform.build_pyramid` and, from the coarsest level to the finest, optimizes a transform+model on the loss of each image warped onto the other. The built-in models return a ``(1, 3, 3)`` destinationāsource+matrix in normalized :math:`[-1, 1]` coordinates (``align_corners=False``): the ``src_homo_dst`` of+:func:`~kornia.geometry.transform.homography_warp` and :class:`~kornia.geometry.transform.HomographyWarper`.++Which class to use:++- Register two images: :class:`ImageRegistrator`, with ``model_type`` ``'homography'``, ``'similarity'``,+ ``'translation'``, ``'rotation'`` or ``'scale'``.+- The models it optimizes: :class:`Homography` (a full homography) and :class:`Similarity` (a rotation, a scale and a+ shift, each one optional). A custom model subclasses :class:`BaseModel` and comes with a warper class, such as a+ subclass of :class:`~kornia.geometry.transform.BaseWarper`.+- To estimate a homography from matched keypoints rather than from pixel values, use+ :class:`~kornia.geometry.ransac.RANSAC` or :func:`~kornia.geometry.homography.find_homography_dlt`.+"""+from abc import abstractmethodfrom typing import Any, Callable, Dict, List, Optional, Tuple, Type, Union@@ -33,7 +52,22 @@ __all__ = ["BaseModel", "Homography", "ImageRegistrator", "Similarity"]class BaseModel(nn.Module):- """Provide an abstract base class for image registration models."""+ """Declare the interface of a transform model that ImageRegistrator optimizes.++ A subclass holds its transform parameters as :class:`~torch.nn.parameter.Parameter` and implements+ ``reset_model()``, which returns them to the identity, ``forward()``, the matrix with which its warper warps the+ source image into the destination, and ``forward_inverse()``, the matrix of the reverse warp.+ :class:`ImageRegistrator` takes an instance as ``model_type``, together with a ``warper`` class. The three methods+ are marked abstract, but :class:`torch.nn.Module` does not enforce it: ``BaseModel`` itself constructs, as+ :class:`~kornia.geometry.transform.BaseWarper` does (`#6399 <https://github.com/kornia/kornia/issues/6399>`_). For+ the built-in models, use :class:`Homography` or :class:`Similarity`.++ See Also:+ - :class:`Homography`: the built-in model of a full homography.+ - :class:`Similarity`: the built-in model of a rotation, a scale and a shift.+ - :class:`ImageRegistrator`: optimizes a model of this interface.++ """@abstractmethoddef reset_model(self) -> None:@@ -63,7 +97,19 @@ class BaseModel(nn.Module):class Homography(BaseModel):- r"""Homography geometric model to be used with ImageRegistrator for the optimization-based image registration."""+ r"""Hold a learnable 3x3 homography for ImageRegistrator, starting from the identity.++ ``model = Homography()`` holds one ``(3, 3)`` :class:`~torch.nn.parameter.Parameter`, initialized to the+ identity. ``model()`` returns it as a ``(1, 3, 3)`` matrix scaled so that its ``[2, 2]`` entry is 1, and+ ``model.forward_inverse()`` its inverse, up to scale. ``ImageRegistrator('homography')`` optimizes it as the+ destinationāsource homography in normalized coordinates. For a rotation, a scale and a shift only, use+ :class:`Similarity`.++ See Also:+ - :class:`Similarity`: four degrees of freedom instead of eight, each of rotation, scale and shift optional.+ - :class:`ImageRegistrator`: optimizes this model with ``model_type='homography'``.++ """def __init__(self) -> None:super().__init__()@@ -97,7 +143,13 @@ class Homography(BaseModel):class Similarity(BaseModel):- """Similarity geometric model to be used with ImageRegistrator module for the optimization-based image registration.+ """Hold a learnable rotation, scale and shift for ImageRegistrator, each one optional.++ ``model = Similarity(rotation, scale, shift)`` holds an angle ``rot`` in degrees, a ``scale`` and a ``(1, 2, 1)``+ ``shift`` in normalized units, each a :class:`~torch.nn.parameter.Parameter` when its flag is ``True`` and+ otherwise a constant buffer (0, 1 and 0). ``model()`` returns the ``(1, 3, 3)`` matrix described below and+ ``model.forward_inverse()`` its inverse. :class:`ImageRegistrator` uses it for ``'similarity'`` (all three),+ ``'translation'`` (the shift), ``'rotation'`` and ``'scale'``. For a full homography, use :class:`Homography`.Convention:- ``forward()`` is ``[[scale * R, shift], [0, 0, 1]]`` in the normalized :math:`[-1, 1]` coordinates of@@ -117,6 +169,10 @@ class Similarity(BaseModel):scale: if True, the scale is optimizable, else constant one.shift: if True, the shift is optimizable, else constant zero.+ See Also:+ - :class:`Homography`: a full homography, eight degrees of freedom.+ - :class:`ImageRegistrator`: optimizes this model for every ``model_type`` but ``'homography'``.+"""def __init__(self, rotation: bool = True, scale: bool = True, shift: bool = True) -> None:@@ -185,7 +241,16 @@ class Similarity(BaseModel):class ImageRegistrator(nn.Module):- r"""nn.Module, which performs optimization-based image registration.+ r"""Register two images by optimizing a transform model on their Gaussian pyramids, coarse to fine.++ Build ``registrator = ImageRegistrator(model_type)``, then ``M = registrator.register(src_img, dst_img)`` takes two+ ``(1, C, H, W)`` images (the built-in models hold one transform, and a larger batch raises). From the coarsest of+ ``pyramid_levels`` Gaussian levels to the finest, it runs up to ``num_iterations`` steps of ``optimizer`` on the+ masked loss of ``src_img`` warped onto ``dst_img`` plus ``dst_img`` warped back onto ``src_img``, and returns the+ model's matrix, ``(1, 3, 3)`` for the built-in models (with the matrix after each level as well, when+ ``output_intermediate_models=True``); :meth:`warp_src_into_dst` then warps an image by it. With+ more than one level, both sides need at least ``3 * 2 ** (pyramid_levels - 2)`` pixels, 24 for the default 5. To+ align images from matched keypoints instead, use :class:`~kornia.geometry.ransac.RANSAC`.Convention:- :meth:`register` returns ``self.model()`` unchanged. For the built-in string ``model_type`` values, this@@ -217,6 +282,12 @@ class ImageRegistrator(nn.Module):``dst_img``, and a different batch or channel size still raises ``ValueError``; if False, images ofdifferent shapes raise ``ValueError``.+ See Also:+ - :class:`Homography`: the model of ``model_type='homography'``.+ - :class:`Similarity`: the model of the other string ``model_type`` values.+ - :class:`~kornia.geometry.transform.HomographyWarper`: the warper of the built-in models.+ - :class:`~kornia.geometry.ransac.RANSAC`: a homography from matched keypoints, robust to outliers.+Example:>>> from kornia.geometry import ImageRegistrator>>> img_src = torch.rand(1, 1, 32, 32)diff --git a/kornia/geometry/transform/imgwarp.py b/kornia/geometry/transform/imgwarp.pyindex 839984d46..f0e2d781f 100644--- a/kornia/geometry/transform/imgwarp.py+++ b/kornia/geometry/transform/imgwarp.py@@ -15,6 +15,37 @@# limitations under the License.#+"""Warp images and volumes by matrices or sampling maps, and build the matrices the warps take.++The matrix warps take channels-first images ``(B, C, H, W)`` or volumes ``(B, C, D, H, W)`` and return the size+``dsize``, ``(h, w)`` or ``(d, h, w)``. Their matrix maps **source** pixel coordinates to **destination** pixel+coordinates, with pixel centres at integers and ``x`` the column, except that :func:`homography_warp` and+:func:`homography_warp3d` take a destinationāsource homography, in normalized :math:`[-1, 1]` coordinates by default.+``align_corners`` defaults differ between these functions, and ``warp_affine3d``, ``warp_perspective3d`` and+``homography_warp3d`` in normalized coordinates are correct only at ``align_corners=True``+(`#4503 <https://github.com/kornia/kornia/issues/4503>`_).++Which function to use:++- An image by a pixel homography or affine matrix: :func:`warp_perspective` ``(B, 3, 3)`` or :func:`warp_affine`+ ``(B, 2, 3)``; a volume: :func:`warp_perspective3d` ``(B, 4, 4)`` or :func:`warp_affine3d` ``(B, 3, 4)``.+- An image or a volume by a destinationāsource homography in normalized coordinates: :func:`homography_warp`,+ :func:`homography_warp3d`.+- An image at given per-pixel source coordinates: :func:`remap`. The warped sampling grid alone, without sampling:+ :func:`warp_grid`, :func:`warp_grid3d`.+- A homography from four point pairs: :func:`get_perspective_transform`; from 3D box corners:+ :func:`get_perspective_transform3d`.+- A 2D affine matrix from parameters: rotation and scale :func:`get_rotation_matrix2d`, translation+ :func:`get_translation_matrix2d`, shear :func:`get_shear_matrix2d`, all of them :func:`get_affine_matrix2d`.+- A 3D affine matrix from parameters: rotation and scale :func:`get_projective_transform`, shear+ :func:`get_shear_matrix3d`, all of them :func:`get_affine_matrix3d`.+- The inverse of a ``(B, 2, 3)`` affine matrix: :func:`invert_affine_transform`. ``[R | t]`` from a rotation and a+ translation: :func:`projection_from_Rt`.+- To rotate, shift, scale or shear an image by parameters and keep its size, without building a matrix, use+ :func:`~kornia.geometry.transform.rotate`, :func:`~kornia.geometry.transform.translate`,+ :func:`~kornia.geometry.transform.scale` or :func:`~kornia.geometry.transform.shear`.+"""+from __future__ import annotationsfrom typing import Optional@@ -260,10 +291,17 @@ def warp_perspective(align_corners: bool = True,fill_value: Optional[torch.Tensor] = None, # needed for jit) -> torch.Tensor:- r"""Warp a 2D (B, C, H, W) image by a (B, 3, 3) sourceādestination homography in pixel coordinates.+ r"""Warp an image by a sourceādestination homography in pixel coordinates.++ ``out = warp_perspective(src, M, dsize)`` takes images ``(B, C, H, W)`` and one ``(B, 3, 3)`` homography per image+ that maps **source** pixels to **destination** pixels, and returns the ``(B, C, h, w)`` destination images for+ ``dsize = (h, w)``. Pixel centres are integers, so an identity ``M`` with a smaller ``dsize`` returns the top-left+ ``dsize`` window of the image, not a resized image. This is the warp for the homographies that+ :func:`get_perspective_transform`, :func:`~kornia.geometry.homography.find_homography_dlt` and+ :class:`~kornia.geometry.ransac.RANSAC` return; for a destinationāsource homography in normalized :math:`[-1, 1]`+ coordinates, use :func:`homography_warp`, and for a ``(B, 2, 3)`` affine matrix, :func:`warp_affine`.- The function warp_perspective transforms the source image using- the specified matrix:+ Each output pixel ``(x, y)`` samples the source at the inverse of ``M`` applied to it:.. math::\text{dst} (x, y) = \text{src} \left(@@ -275,11 +313,14 @@ def warp_perspective(See :doc:`Conventions & Pitfalls </get-started/conventions>` for transform direction, pixel centres,normalized coordinates and ``align_corners`` sampling rules.- - input: ``(B, C, H, W)``; ``dsize`` is ``(h, w)``- ``M`` is the sourceādestination **pixel** homography ``(B, 3, 3)``(contrast :func:`homography_warp`, which by default consumes destinationāsource normalized)+ - ``M`` has the batch of ``src``: a ``(1, 3, 3)`` ``M`` is not shared by a larger batch, as it is in+ :func:`warp_affine`- a nonzero scalar multiple of ``M`` describes the same warp; uniformly small or largenonzero entries are rescaled toward unit magnitude before the coordinate change and inversion+ - ``src`` sets the dtype: a float32 ``M`` with a float64 ``src`` samples in float64, and a float64 ``M`` with a+ float32 ``src`` raises ``RuntimeError``- a zero output dimension returns an autograd-connected empty tensor;negative output dimensions raise ``ValueError``@@ -292,11 +333,22 @@ def warp_perspective(| ``'fill'``.align_corners: interpolation flag.fill_value: torch.Tensor of shape ``(C)`` or ``(1)`` that fills the padding area, or any shape that- broadcasts to the output :math:`(B, C, h, w)` without changing it, such as a per-sample ``(B, C, 1, 1)``+ broadcasts to the output ``(B, C, h, w)`` without changing it, such as a per-sample ``(B, C, 1, 1)``fill. Defaults to zeros.Returns:- the warped input image :math:`(B, C, h, w)`, spatial sizes given by ``dsize``.+ the warped input image ``(B, C, h, w)``, spatial sizes given by ``dsize``.++ See Also:+ - :func:`homography_warp`: a destinationāsource homography, normalized to :math:`[-1, 1]` by default.+ - :func:`warp_perspective3d`: the same warp for ``(B, C, D, H, W)`` volumes by ``(B, 4, 4)`` homographies.+ - :func:`warp_affine`: a ``(B, 2, 3)`` affine matrix instead, where a single ``(1, 2, 3)`` one serves a batch.+ - :func:`get_perspective_transform`: the homography that maps four source points onto four destination points.+ - :func:`remap`: sample at given per-pixel source coordinates instead of by a matrix.+ - :func:`crop_by_transform_mat`: dispatches to this warp or :func:`warp_affine` by matrix shape, casting the+ matrix to the image dtype and broadcasting a batch-1 matrix.+ - :func:`~kornia.geometry.conversions.normalize_homography`: ``M`` in normalized :math:`[-1, 1]` coordinates,+ the conversion this warp applies before sampling.Example:>>> img = torch.rand(1, 4, 5, 6)@@ -305,9 +357,6 @@ def warp_perspective(>>> print(out.shape)torch.Size([1, 4, 4, 2])- .. note::- This function is often used in conjunction with :func:`get_perspective_transform`.-.. note::See a working example `here <https://www.kornia.org/tutorials/nbs/warp_perspective.html>`_.@@ -391,12 +440,19 @@ def warp_affine(align_corners: bool = True,fill_value: Optional[torch.Tensor] = None,) -> torch.Tensor:- r"""Apply an affine transformation to a torch.Tensor.+ r"""Warp an image by a sourceādestination affine matrix in pixel coordinates.++ ``out = warp_affine(src, M, dsize)`` takes images ``(B, C, H, W)`` and ``(B, 2, 3)`` affine matrices that map+ **source** pixels to **destination** pixels, and returns the ``(B, C, h, w)`` destination images for+ ``dsize = (h, w)``. A single ``(1, 2, 3)`` matrix serves every image of a non-empty batch. Build ``M`` with+ :func:`get_rotation_matrix2d`, or take the top two rows ``M[:, :2]`` of the ``(B, 3, 3)`` matrix of+ :func:`get_affine_matrix2d`; :func:`invert_affine_transform` gives the matrix that warps back. To keep the input+ size, use :func:`affine`, or pass an angle, a shift or a scale instead of a matrix to :func:`rotate`,+ :func:`translate` or :func:`scale`; for a ``(B, 3, 3)`` homography, use :func:`warp_perspective`... image:: _static/img/warp_affine.png- The function warp_affine transforms the source torch.Tensor using- the specified matrix:+ Each output pixel ``(x, y)`` samples the source at:.. math::\text{dst}(x, y) = \text{src} \left( M^{-1}_{11} x + M^{-1}_{12} y + M^{-1}_{13} ,@@ -408,32 +464,36 @@ def warp_affine(See :doc:`Conventions & Pitfalls </get-started/conventions>` for transform direction, pixel centres,normalized coordinates and ``align_corners`` sampling rules.- - input: ``(B, C, H, W)``; ``dsize`` is ``(h, w)``- ``M`` is the sourceādestination **pixel** affine matrix ``(B, 2, 3)``+ - ``src`` sets the dtype: a float32 ``M`` with a float64 ``src`` samples in float64, and a float64 ``M`` with a+ float32 ``src`` raises ``RuntimeError``- a zero output dimension returns an autograd-connected empty tensor;negative output dimensions raise ``ValueError``Args:src: input torch.Tensor of shape ``(B, C, H, W)``.- M: affine transformation of shape ``(B, 2, 3)``.+ M: affine transformation of shape ``(B, 2, 3)``, or ``(1, 2, 3)`` for the whole batch.dsize: size of the output image (height, width).mode: interpolation mode to calculate output values ``'bilinear'`` | ``'nearest'``.padding_mode: padding mode for outside grid values ``'zeros'`` | ``'border'`` | ``'reflection'``| ``'fill'``.align_corners : mode for grid_generation.fill_value: torch.Tensor of shape ``(C)`` or ``(1)`` that fills the padding area, or any shape that- broadcasts to the output :math:`(B, C, h, w)` without changing it, such as a per-sample ``(B, C, 1, 1)``- fill.+ broadcasts to the output ``(B, C, h, w)`` without changing it, such as a per-sample ``(B, C, 1, 1)``+ fill. Defaults to zeros.Returns:- the warped torch.Tensor with shape :math:`(B, C, h, w)`, spatial sizes given by ``dsize``.-- .. note::- This function is often used in conjunction with :func:`get_rotation_matrix2d`,- :func:`get_shear_matrix2d`, :func:`get_affine_matrix2d`, :func:`invert_affine_transform`.-- .. note::- See a working example `here <https://www.kornia.org/tutorials/nbs/rotate_affine.html>`__.+ the warped torch.Tensor with shape ``(B, C, h, w)``, spatial sizes given by ``dsize``.++ See Also:+ - :func:`warp_perspective`: a ``(B, 3, 3)`` homography per image, for perspective warps.+ - :func:`warp_affine3d`: the same warp for ``(B, C, D, H, W)`` volumes by ``(B, 3, 4)`` matrices.+ - :func:`affine`: keeps the input size and also takes an unbatched ``(C, H, W)`` image.+ - :func:`invert_affine_transform`: the destinationāsource matrix, to warp back.+ - :func:`get_affine_matrix2d`: a ``(B, 3, 3)`` matrix from translation, rotation, scale and shear; pass its+ ``M[:, :2]``.+ - :func:`crop_by_transform_mat`: calls this warp for a ``(B, 2, 3)`` matrix after casting it to the image dtype,+ and :func:`warp_perspective` for a ``(B, 3, 3)`` one.Example:>>> img = torch.rand(1, 4, 5, 6)@@ -442,6 +502,9 @@ def warp_affine(>>> print(out.shape)torch.Size([1, 4, 4, 2])+ .. note::+ See a working example `here <https://www.kornia.org/tutorials/nbs/rotate_affine.html>`__.+"""if not isinstance(src, torch.Tensor):raise TypeError(f"Input src type is not a torch.Tensor. Got {type(src)}")@@ -598,14 +661,21 @@ def _fill_and_warp(def warp_grid(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:- r"""Transform a 2D (x, y) coordinate grid by destinationāsource (N, 3, 3) homographies.+ """Warp a grid of 2D points by homographies, returning the coordinates without sampling an image.++ ``warped = warp_grid(grid, src_homo_dst)`` applies the homographies ``src_homo_dst`` to every ``(x, y)`` point of+ ``grid`` and returns the ``(N, H, W, 2)`` points after the homogeneous division, in the dtype of ``src_homo_dst``.+ It does not normalize: the points stay in the coordinates that ``grid`` and the homography share, so a normalized+ grid from :func:`~kornia.geometry.grid.create_meshgrid` and a normalized homography give positions that+ :func:`torch.nn.functional.grid_sample` reads directly. To warp an image, use :func:`homography_warp`, which builds+ such a grid, warps it with this function and samples the image.Convention:- ``grid`` coordinates: ``(x, y)`` (last dim), shape ``(1, H, W, 2)`` or ``(N, H, W, 2)``- ``src_homo_dst`` is the destinationāsource homography ``(1, 3, 3)``,``(N, 3, 3)`` or ``(N, 1, 3, 3)``- only the ``(1, H, W, 2)`` grid broadcasts over homography batches; a batched grid- requires a matching batch of :math:`N` homographies+ requires a matching batch of ``N`` homographies- ``src_homo_dst`` is divided by a detached scale toward unit magnitude before thehomogeneous division, as in :func:`warp_perspective`, so a nonzero scalar multiple of itreturns the same grid, except at grid points whose homogeneous coordinate is within@@ -615,7 +685,7 @@ def warp_grid(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:Args:grid: Unwrapped grid of the shape ``(1, H, W, 2)``, or ``(N, H, W, 2)`` with a- matching batch of :math:`N` homographies.+ matching batch of ``N`` homographies.src_homo_dst: Homography or homographies (stacked) totransform all points in the grid. Shape of the homographyhas to be ``(1, 3, 3)``, ``(N, 3, 3)`` or ``(N, 1, 3, 3)``.@@ -623,6 +693,13 @@ def warp_grid(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:Returns:the transformed grid of shape ``(N, H, W, 2)``.+ See Also:+ - :func:`warp_grid3d`: the same for a 3D ``(x, y, z)`` grid and ``(N, 4, 4)`` homographies.+ - :func:`homography_warp`: builds the grid, warps it with this function and samples the image.+ - :func:`~kornia.geometry.grid.create_meshgrid`: the ``(1, H, W, 2)`` grid to pass in, in pixels or normalized.+ - :func:`~kornia.geometry.linalg.transform_points`: transform any ``(B, N, 2)`` point set by ``(B, 3, 3)``+ matrices.+"""batch_size: int = src_homo_dst.size(0)_, height, width, _ = grid.size()@@ -684,7 +761,14 @@ def _normalize_warped_pixel_grid(grid: torch.Tensor, height: int, width: int, aldef warp_grid3d(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:- r"""Transform a 3D (x, y, z) coordinate grid by destinationāsource (N, 4, 4) homographies.+ """Warp a grid of 3D points by 4x4 homographies, returning the coordinates without sampling a volume.++ ``warped = warp_grid3d(grid, src_homo_dst)`` applies the homographies ``src_homo_dst`` to every ``(x, y, z)`` point+ of ``grid`` and returns the ``(N, D, H, W, 3)`` points after the homogeneous division, in the dtype of+ ``src_homo_dst``. It does not normalize or reorder: the last axis must already be ``(x, y, z)``, the order the+ homography and :func:`torch.nn.functional.grid_sample` read, whereas+ :func:`~kornia.geometry.grid.create_meshgrid3d` returns ``(d, x, y)``. To warp a volume, use+ :func:`homography_warp3d`, which builds and reorders the grid, warps it with this function and samples the volume.Convention:- ``grid`` coordinates: ``(x, y, z)`` (last dim), shape ``(1, D, H, W, 3)`` or@@ -692,7 +776,7 @@ def warp_grid3d(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:- ``src_homo_dst`` is the destinationāsource homography ``(1, 4, 4)``,``(N, 4, 4)`` or ``(N, 1, 4, 4)``- only the ``(1, D, H, W, 3)`` grid broadcasts over homography batches; a batched- grid requires a matching batch of :math:`N` homographies+ grid requires a matching batch of ``N`` homographies- ``src_homo_dst`` is divided by a detached scale toward unit magnitude before thehomogeneous division, as in :func:`warp_perspective`, so a nonzero scalar multiple of itreturns the same grid, except at grid points whose homogeneous coordinate is within@@ -702,7 +786,7 @@ def warp_grid3d(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:Args:grid: Unwrapped grid of the shape ``(1, D, H, W, 3)``, or ``(N, D, H, W, 3)``- with a matching batch of :math:`N` homographies.+ with a matching batch of ``N`` homographies.src_homo_dst: Homography or homographies (stacked) totransform all points in the grid. Shape of the homographyhas to be ``(1, 4, 4)``, ``(N, 4, 4)`` or ``(N, 1, 4, 4)``.@@ -710,6 +794,12 @@ def warp_grid3d(grid: torch.Tensor, src_homo_dst: torch.Tensor) -> torch.Tensor:Returns:the transformed grid of shape ``(N, D, H, W, 3)``.+ See Also:+ - :func:`warp_grid`: the same for a 2D ``(x, y)`` grid and ``(N, 3, 3)`` homographies.+ - :func:`homography_warp3d`: builds and reorders the grid, warps it with this function and samples the volume.+ - :func:`~kornia.geometry.grid.create_meshgrid3d`: a grid in ``(d, x, y)`` order, to reorder before passing it+ in.+"""batch_size: int = src_homo_dst.size(0)_, depth, height, width, _ = grid.size()@@ -846,13 +936,17 @@ def _get_perspective_transform_closed_form(points_src: torch.Tensor, points_dst:def get_perspective_transform(points_src: torch.Tensor, points_dst: torch.Tensor) -> torch.Tensor:- r"""Calculate a perspective transform from four pairs of the corresponding points.+ r"""Compute the homography that maps four source points exactly onto four destination points.- The algorithm is a vanilla implementation of the Direct Linear transform (DLT).- See more: https://www.cs.cmu.edu/~16385/s17/Slides/10.2_2D_Alignment__DLT.pdf+ ``M = get_perspective_transform(points_src, points_dst)`` takes two ``(B, 4, 2)`` sets of ``(x, y)`` points,+ matched by index, and returns the ``(B, 3, 3)`` homography, scaled to ``M[2, 2] = 1``, that maps each source point+ onto its destination point: the sourceādestination pixel homography that :func:`warp_perspective` takes. It solves+ the four-point system in closed form, by Heckbert's square-to-quadrilateral mappings, rather than with the+ `DLT <https://www.cs.cmu.edu/~16385/s17/Slides/10.2_2D_Alignment__DLT.pdf>`_, whose exact solution it equals, so it+ needs no linear solver. For more than four or noisy correspondences, use+ :func:`~kornia.geometry.homography.find_homography_dlt` or :class:`~kornia.geometry.ransac.RANSAC`.- The function calculates the matrix of a perspective transform that maps from- the source to destination points:+ The matrix maps each source point onto its destination point, up to the projective scale:.. math::@@ -876,8 +970,16 @@ def get_perspective_transform(points_src: torch.Tensor, points_dst: torch.TensorConvention:- points: ``(x, y)``, pixel centers, origin at top-left; shape ``(B, 4, 2)``- - returns the sourceādestination **pixel** homography ``(B, 3, 3)`` that- :func:`warp_perspective` takes+ - a degenerate quadrilateral, such as one with three collinear points or two equal ones, does not raise: a+ degenerate source returns a non-finite matrix, a degenerate destination a non-finite or a singular one+ - a homography whose true bottom-right entry is zero, such as one that sends source pixel ``(0, 0)`` to+ infinity, has no ``M[2, 2] = 1`` form, although neither quadrilateral is degenerate: the division by+ ``M[2, 2]`` gives NaN or inf when rounding leaves that entry exactly zero, and otherwise a finite matrix+ scaled by the inverse of the rounding residue, whose largest entry is typically ``1e5`` in ``float32`` and+ ``1e13`` in ``float64`` and which still maps the points, to a relative error of up to about ``1e-2`` in+ ``float32`` (`#6435 <https://github.com/kornia/kornia/issues/6435>`_);+ :func:`~kornia.geometry.homography.find_homography_dlt` returns it, at another scale+ - float16 and bfloat16 points are solved in float32, and the result is cast back to their dtype- a caller's autocast region does not change the result, outside TorchScriptArgs:@@ -888,8 +990,12 @@ def get_perspective_transform(points_src: torch.Tensor, points_dst: torch.TensorReturns:the perspective transformation with shape ``(B, 3, 3)``.- .. note::- This function is often used in conjunction with :func:`warp_perspective`.+ See Also:+ - :func:`warp_perspective`: warp an image by the returned homography.+ - :func:`get_perspective_transform3d`: the 3D counterpart, a ``(B, 4, 4)`` homography from box corners.+ - :func:`~kornia.geometry.homography.find_homography_dlt`: a least-squares homography from any number of+ weighted point pairs.+ - :class:`~kornia.geometry.ransac.RANSAC`: a homography that is robust to wrong correspondences.Example:>>> x1 = torch.tensor([[[0., 0.], [1., 0.], [1., 1.], [0., 1.]]])@@ -916,9 +1022,15 @@ def get_perspective_transform(points_src: torch.Tensor, points_dst: torch.Tensor# TODO: move to kornia.geometry.affinedef get_rotation_matrix2d(center: torch.Tensor, angle: torch.Tensor, scale: torch.Tensor) -> torch.Tensor:- r"""Calculate an affine matrix of 2D rotation.+ r"""Compute the 2D affine matrix that scales and rotates counter-clockwise, in degrees, about a centre.- The function calculates the following matrix:+ ``M = get_rotation_matrix2d(center, angle, scale)`` takes centres ``(B, 2)`` as ``(x, y)`` pixels, angles ``(B,)``+ in degrees and scale factors ``(B, 2)`` for ``x`` and ``y``, and returns the ``(B, 2, 3)`` sourceādestination pixel+ matrix that scales and then rotates about ``center``, which stays fixed. A positive angle turns counter-clockwise as+ displayed, with ``y`` pointing down. Pass ``M`` to :func:`warp_affine`; to add a translation and a shear in one+ ``(B, 3, 3)`` matrix, use :func:`get_affine_matrix2d`, and to rotate an image directly, :func:`rotate`.++ With equal scale factors ``scale``, the matrix is:.. math::\begin{bmatrix}@@ -931,11 +1043,10 @@ def get_rotation_matrix2d(center: torch.Tensor, angle: torch.Tensor, scale: torcwhere.. math::- \alpha = \text{scale} \cdot torch.cos(\text{angle}) \\- \beta = \text{scale} \cdot torch.sin(\text{angle})+ \alpha = \text{scale} \cdot \cos(\text{angle}) \\+ \beta = \text{scale} \cdot \sin(\text{angle})- The transformation maps the rotation center to itself- If this is not the target, adjust the shift.+ With two different factors, the left 2x2 block is the rotation times ``diag(scale[:, 0], scale[:, 1])``.Convention:- ``center`` is ``(x, y)`` in pixels, origin at top-left@@ -956,6 +1067,13 @@ def get_rotation_matrix2d(center: torch.Tensor, angle: torch.Tensor, scale: torcTypeError: If any input is not a tensor.ValueError: If input shapes are invalid, or batch sizes, devices or dtypes do not match.+ See Also:+ - :func:`get_affine_matrix2d`: also translates and shears, returns ``(B, 3, 3)``, and turns clockwise for a+ positive angle (`#4408 <https://github.com/kornia/kornia/issues/4408>`_).+ - :func:`rotate`: rotate an image about its centre without building the matrix.+ - :func:`get_projective_transform`: the 3D rotation and scale, by an axis-angle vector.+ - :func:`~kornia.geometry.conversions.angle_to_rotation_matrix`: the 2x2 rotation matrix of an angle alone.+Example:>>> center = torch.zeros(1, 2)>>> scale = torch.ones((1, 2))@@ -964,9 +1082,6 @@ def get_rotation_matrix2d(center: torch.Tensor, angle: torch.Tensor, scale: torctensor([[[ 0.7071, 0.7071, 0.0000],[-0.7071, 0.7071, 0.0000]]])- .. note::- This function is often used in conjunction with :func:`warp_affine`.-"""if not isinstance(center, torch.Tensor):raise TypeError(f"Input center type is not a torch.Tensor. Got {type(center)}")@@ -1028,18 +1143,23 @@ def remap(align_corners: Optional[bool] = None,normalized_coordinates: bool = False,) -> torch.Tensor:- r"""Apply a generic geometrical transformation to an image torch.Tensor.+ r"""Sample an image at per-pixel source coordinates given as an x map and a y map.- .. image:: _static/img/remap.png+ ``out = remap(image, map_x, map_y)`` takes images ``(B, C, H, W)`` and two ``(B, h, w)`` maps that hold, for every+ output pixel, the source column ``x`` and row ``y`` to sample, and returns ``(B, C, h, w)``. The maps are absolute+ pixel coordinates, not displacements, so the identity maps are the two channels of+ :func:`~kornia.geometry.grid.create_meshgrid` with ``normalized_coordinates=False``; with+ ``normalized_coordinates=True`` they are the :math:`[-1, 1]` coordinates of+ :func:`torch.nn.functional.grid_sample`. Maps of batch 1 serve every image. With pixel maps, pass+ ``align_corners=True``: the default resamples even an identity map. For a warp given by a matrix, use+ :func:`warp_affine` or :func:`warp_perspective`.- The function remap transforms the source torch.Tensor using the specified map:+ .. image:: _static/img/remap.png.. math::\text{dst}(x, y) = \text{src}(map_x(x, y), map_y(x, y))Convention:- - input: ``(B, C, H, W)``; ``map_x``/``map_y`` are ``(B, H, W)`` pixel coordinates- unless ``normalized_coordinates=True``- align_corners: ``None`` by default, resolved to ``False`` internally- pixel maps are normalized with the ``align_corners=True`` convention whatever flagreaches ``grid_sample``, so at ``False``/``None`` even an identity map resamples the image;@@ -1048,12 +1168,11 @@ def remap(autograd-connected empty output, including when the matching source axis is emptyArgs:- image: the torch.Tensor to remap with shape (B, C, H, W).+ image: the torch.Tensor to remap with shape ``(B, C, H, W)``.Where C is the number of channels.- map_x: the flow in the x-direction in pixel coordinates.- The torch.Tensor must be in the shape of (B, H, W).- map_y: the flow in the y-direction in pixel coordinates.- The torch.Tensor must be in the shape of (B, H, W).+ map_x: the source x coordinate (column) to sample for each output pixel, with shape ``(B, h, w)`` or+ ``(1, h, w)``; in pixels unless ``normalized_coordinates=True``.+ map_y: the source y coordinate (row) to sample for each output pixel, with the shape of ``map_x``.mode: interpolation mode to calculate output values``'bilinear'`` | ``'nearest'``.padding_mode: padding mode for outside grid values@@ -1063,7 +1182,13 @@ def remap(normalized in the range of [-1, 1].Returns:- the warped torch.Tensor with same shape as the input grid maps.+ the sampled images ``(B, C, h, w)``, with the spatial size of the maps.++ See Also:+ - :func:`warp_perspective`: warp by a ``(B, 3, 3)`` pixel homography instead of maps.+ - :func:`warp_grid`: the positions a homography gives, as a grid whose two channels are maps.+ - :func:`~kornia.geometry.grid.create_meshgrid`: the identity maps, to offset or transform.+ - :func:`warp_image_tps`: a smooth warp from thin-plate-spline control points.Example:>>> import torch@@ -1075,9 +1200,6 @@ def remap(tensor([[[[1., 0.],[0., 0.]]]])- .. note::- This function is often used in conjunction with :func:`kornia.geometry.grid.create_meshgrid`.-"""KORNIA_CHECK_SHAPE(image, ["B", "C", "H", "W"])KORNIA_CHECK_SHAPE(map_x, ["B", "H", "W"])@@ -1123,10 +1245,15 @@ def remap(def invert_affine_transform(matrix: torch.Tensor) -> torch.Tensor:- r"""Invert an affine transformation.+ r"""Invert 2D affine matrices of shape ``(B, 2, 3)``.++ ``M_inv = invert_affine_transform(M)`` returns, for each ``(B, 2, 3)`` affine matrix, the ``(B, 2, 3)`` inverse of+ its 3x3 extension with the last row ``(0, 0, 1)``, in the dtype of ``M``. It is pure matrix inversion, so the+ result keeps the input's coordinates, pixel or normalized: the inverse of a sourceādestination matrix maps+ destination to source, and warps a :func:`warp_affine` result back. For ``(B, 3, 3)`` homographies, use+ :func:`torch.linalg.inv`.- The function computes an inverse affine transformation represented by- 2x3 matrix:+ The input is the matrix:.. math::\begin{bmatrix}@@ -1134,11 +1261,10 @@ def invert_affine_transform(matrix: torch.Tensor) -> torch.Tensor:a_{21} & a_{22} & b_{2} \\\end{bmatrix}- The result is also a 2x3 matrix of the same type as M.-Convention:- ``matrix`` is a ``(B, 2, 3)`` affine transform in any coordinate convention(pixel or normalized) ā pure matrix inversion; the result stays in the input's convention+ - in eager mode a singular matrix raises ``torch.linalg.LinAlgError``Args:matrix: original affine transform. The torch.Tensor must be@@ -1147,8 +1273,12 @@ def invert_affine_transform(matrix: torch.Tensor) -> torch.Tensor:Return:the reverse affine transform with shape ``(B, 2, 3)``.- .. note::- This function is often used in conjunction with :func:`warp_affine`.+ See Also:+ - :func:`warp_affine`: warp an image by a matrix or by its inverse.+ - :func:`get_affine_matrix2d`: a ``(B, 3, 3)`` affine matrix from translation, rotation, scale and shear;+ pass its ``M[:, :2]``.+ - :func:`~kornia.geometry.conversions.convert_affinematrix_to_homography`: the ``(B, 3, 3)`` extension, to+ compose with homographies."""if not isinstance(matrix, torch.Tensor):@@ -1171,7 +1301,15 @@ def get_affine_matrix2d(sx: Optional[torch.Tensor] = None,sy: Optional[torch.Tensor] = None,) -> torch.Tensor:- r"""Compose affine matrix from the components.+ """Compose a 2D affine matrix from a translation, clockwise rotation, scale and shear about a centre.++ ``M = get_affine_matrix2d(translations, center, scale, angle, sx, sy)`` takes ``(B, 2)`` translations+ ``(dx, dy)``, centres ``(x, y)`` and scale factors for ``x`` and ``y``, ``(B,)`` angles in degrees and optional+ ``(B,)`` shear angles ``sx`` and ``sy`` in radians, and returns the ``(B, 3, 3)`` sourceādestination pixel matrix+ that shears, scales, rotates and then translates, the first three about ``center``. A positive ``angle`` turns+ **clockwise** as displayed, unlike :func:`get_rotation_matrix2d`, :func:`rotate` and :class:`Affine`+ (`#4408 <https://github.com/kornia/kornia/issues/4408>`_). Pass ``M[:, :2]`` to :func:`warp_affine` or ``M`` to+ :func:`warp_perspective`.Convention:- ``center`` is ``(x, y)`` in pixels, origin at top-left@@ -1192,8 +1330,15 @@ def get_affine_matrix2d(Returns:the affine transformation matrix ``(B, 3, 3)``.- .. note::- This function is often used in conjunction with :func:`warp_affine`, :func:`warp_perspective`.+ See Also:+ - :func:`get_rotation_matrix2d`: rotation and scale only, ``(B, 2, 3)``, counter-clockwise for a positive+ angle.+ - :func:`get_affine_matrix3d`: the 3D composition, ``(B, 4, 4)``, whose positive ``z`` angle turns+ counter-clockwise on screen.+ - :class:`Affine`: warp an image by these parameters in one module, counter-clockwise for a positive angle.+ - :func:`get_shear_matrix2d`: the shear part alone.+ - :func:`~kornia.geometry.solvers.find_affine`: an affine matrix fitted to point correspondences instead of+ built from parameters."""transform: torch.Tensor = get_rotation_matrix2d(center, -angle, scale)@@ -1210,7 +1355,12 @@ def get_affine_matrix2d(def get_translation_matrix2d(translations: torch.Tensor) -> torch.Tensor:- r"""Compose translation matrix from the components.+ """Compute the 3x3 matrix of a 2D translation in pixels.++ ``M = get_translation_matrix2d(translations)`` takes ``(B, 2)`` shifts ``(dx, dy)`` in pixels and returns the+ ``(B, 3, 3)`` matrix that moves every point by them, a positive ``dx`` to the right and a positive ``dy`` down.+ Pass ``M[:, :2]`` to :func:`warp_affine` or ``M`` to :func:`warp_perspective`; to shift an image directly, use+ :func:`translate`.Convention:- ``translations`` is ``(dx, dy)`` in pixels; returns ``(B, 3, 3)`` affine matrix in pixel coordinates@@ -1221,8 +1371,9 @@ def get_translation_matrix2d(translations: torch.Tensor) -> torch.Tensor:Returns:the affine transformation matrix ``(B, 3, 3)``.- .. note::- This function is often used in conjunction with :func:`warp_affine`, :func:`warp_perspective`.+ See Also:+ - :func:`translate`: shift an image without building the matrix.+ - :func:`get_affine_matrix2d`: a translation composed with a rotation, a scale and a shear."""transform: torch.Tensor = eye_like(3, translations)[:, :2, :]@@ -1235,9 +1386,14 @@ def get_translation_matrix2d(translations: torch.Tensor) -> torch.Tensor:def get_shear_matrix2d(center: torch.Tensor, sx: Optional[torch.Tensor] = None, sy: Optional[torch.Tensor] = None) -> torch.Tensor:- r"""Compose shear matrix Bx3x3 from the components.+ r"""Compute the 3x3 matrix of a 2D shear given as two angles in radians about a centre.- Note: Ordered shearing, shear x-axis then y-axis.+ ``M = get_shear_matrix2d(center, sx, sy)`` takes ``(B, 2)`` centres ``(x, y)`` in pixels and optional ``(B,)``+ shear angles ``sx`` and ``sy`` in radians (zero when omitted), and returns the ``(B, 3, 3)`` sourceādestination+ pixel matrix that shears along ``x`` by ``-tan(sx)`` and then along ``y`` by ``-tan(sy)``, both about ``center``.+ :class:`Affine` and :func:`get_affine_matrix2d` read shear angles this way; :func:`shear` and :class:`Shear` take+ raw factors about pixel ``(0, 0)`` instead, which shear the other way for the same values below ``pi / 2`` in+ magnitude (`#5661 <https://github.com/kornia/kornia/issues/5661>`_)... math::\begin{bmatrix}@@ -1260,7 +1416,13 @@ def get_shear_matrix2d(sy: shearing angle along y axis in radiansReturns:- params to be passed to the affine transformation with shape ``(B, 3, 3)``.+ the shear matrix with shape ``(B, 3, 3)``, to be passed to the affine transformation.++ See Also:+ - :func:`get_shear_matrix3d`: the 3D shear from up to six angles, ``(B, 4, 4)``.+ - :func:`shear`: shear an image by raw factors about pixel ``(0, 0)``, which shear the other way below+ ``pi / 2``.+ - :func:`get_affine_matrix2d`: this shear composed with a rotation, a scale and a translation.Examples:>>> rng = torch.manual_seed(0)@@ -1273,9 +1435,6 @@ def get_shear_matrix2d([ -0.0000, 1.0000, 0.0000],[ 0.0000, 0.0000, 1.0000]]])- .. note::- This function is often used in conjunction with :func:`warp_affine`, :func:`warp_perspective`.-"""sx = torch.zeros(center.size(0), device=center.device, dtype=center.dtype) if sx is None else sxsy = torch.zeros(center.size(0), device=center.device, dtype=center.dtype) if sy is None else sy@@ -1305,7 +1464,16 @@ def get_affine_matrix3d(szx: Optional[torch.Tensor] = None,szy: Optional[torch.Tensor] = None,) -> torch.Tensor:- r"""Compose 3d affine matrix from the components.+ """Compose a 3D affine matrix from a translation, axis-angle rotation, scale and shears about a centre.++ ``M = get_affine_matrix3d(translations, center, scale, angles)`` takes ``(B, 3)`` translations ``(dx, dy, dz)``,+ centres ``(x, y, z)`` in pixels, per-axis scale factors and axis-angle vectors in degrees, plus up to six optional+ ``(B,)`` shear angles in radians, and returns the ``(B, 4, 4)`` sourceādestination pixel matrix that shears, scales,+ rotates and then translates, the first three about ``center``. The angles are negated before+ :func:`get_projective_transform`, so they turn the other way from that function and :func:`rotate3d`: a positive+ rotation about ``z`` turns counter-clockwise on screen, as :func:`rotate` does and :func:`get_affine_matrix2d`+ does not (`#4408 <https://github.com/kornia/kornia/issues/4408>`_). Pass ``M[:, :3]`` to :func:`warp_affine3d`+ or ``M`` to :func:`warp_perspective3d`.Convention:- ``center`` is ``(x, y, z)`` in pixels, origin at the top-left of the first depth@@ -1318,22 +1486,28 @@ def get_affine_matrix3d(Args:translations: torch.Tensor containing the translation vector (dx,dy,dz) with shape ``(B, 3)``.center: torch.Tensor containing the center vector (x,y,z) with shape ``(B, 3)``.- scale: torch.Tensor containing the scale factor with shape ``(B)``.+ scale: per-axis scale factors ``(sx, sy, sz)`` with shape ``(B, 3)``; a ``(B, 1)`` or ``(1,)`` tensor scales+ the three axes alike. A ``(B,)`` tensor is not one factor per matrix: for ``B = 3`` it is read as+ ``(sx, sy, sz)``, and for another ``B > 1`` it raises+ (`#6398 <https://github.com/kornia/kornia/issues/6398>`_).angles: axis angle vector containing the rotation angles in degrees in the formof (rx, ry, rz) with shape ``(B, 3)``. Internally it calls Rodrigues to computethe rotation matrix from axis-angle.- sxy: torch.Tensor containing the shear factor in the xy-direction with shape ``(B)``.- sxz: torch.Tensor containing the shear factor in the xz-direction with shape ``(B)``.- syx: torch.Tensor containing the shear factor in the yx-direction with shape ``(B)``.- syz: torch.Tensor containing the shear factor in the yz-direction with shape ``(B)``.- szx: torch.Tensor containing the shear factor in the zx-direction with shape ``(B)``.- szy: torch.Tensor containing the shear factor in the zy-direction with shape ``(B)``.+ sxy: the shear angle ``sxy`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.+ sxz: the shear angle ``sxz`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.+ syx: the shear angle ``syx`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.+ syz: the shear angle ``syz`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.+ szx: the shear angle ``szx`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.+ szy: the shear angle ``szy`` of :func:`get_shear_matrix3d`, in radians, with shape ``(B)``.Returns:the 3d affine transformation matrix ``(B, 4, 4)``.- .. note::- This function is often used in conjunction with :func:`warp_perspective3d`.+ See Also:+ - :func:`get_affine_matrix2d`: the 2D composition, whose positive angle turns clockwise on screen.+ - :func:`get_projective_transform`: rotation and scale only, ``(B, 3, 4)``, with the angles as given.+ - :func:`get_shear_matrix3d`: the shear part alone.+ - :func:`warp_affine3d`: warp a volume by ``M[:, :3]``."""transform: torch.Tensor = get_projective_transform(center, -angles, scale)@@ -1357,9 +1531,14 @@ def get_shear_matrix3d(szx: Optional[torch.Tensor] = None,szy: Optional[torch.Tensor] = None,) -> torch.Tensor:- r"""Compose shear matrix Bx4x4 from the components.+ r"""Compute the 4x4 matrix of a 3D shear given as up to six angles in radians about a centre.- Note: Ordered shearing, shear x-axis then y-axis then z-axis.+ ``M = get_shear_matrix3d(center, sxy, sxz, syx, syz, szx, szy)`` takes ``(B, 3)`` centres ``(x, y, z)`` in pixels+ and optional ``(B,)`` shear angles in radians (zero when omitted), and returns the ``(B, 4, 4)`` sourceādestination+ pixel matrix below, whose translation keeps ``center`` fixed. With :math:`S_{ab} = -\tan(s_{ab})`, the angle ``sab``+ on its own adds :math:`S_{ab}` times coordinate ``a`` to coordinate ``b``. ``sxy`` and ``sxz`` form one shear,+ ``syx`` and ``syz`` a second and ``szx`` and ``szy`` a third, and the linear part is the product of the three in+ that order, so a point is sheared by ``szx`` and ``szy`` first and by ``sxy`` and ``sxz`` last... math::\begin{bmatrix}@@ -1386,16 +1565,21 @@ def get_shear_matrix3d(- returns ``(B, 4, 4)`` affine matrix in pixel coordinatesArgs:- center: shearing center coordinates of (x, y, z).- sxy: shearing angle along x axis, towards y plane in radiants.- sxz: shearing angle along x axis, towards z plane in radiants.- syx: shearing angle along y axis, towards x plane in radiants.- syz: shearing angle along y axis, towards z plane in radiants.- szx: shearing angle along z axis, towards x plane in radiants.- szy: shearing angle along z axis, towards y plane in radiants.+ center: shearing center coordinates of (x, y, z), with shape ``(B, 3)``.+ sxy: shearing angle in radians that adds :math:`S_{xy} x` to ``y``.+ sxz: shearing angle in radians that adds :math:`S_{xz} x` to ``z``.+ syx: shearing angle in radians that adds :math:`S_{yx} y` to ``x``.+ syz: shearing angle in radians that adds :math:`S_{yz} y` to ``z``.+ szx: shearing angle in radians that adds :math:`S_{zx} z` to ``x``.+ szy: shearing angle in radians that adds :math:`S_{zy} z` to ``y``.Returns:- params to be passed to the affine transformation.+ the shear matrix with shape ``(B, 4, 4)``, to be passed to the affine transformation.++ See Also:+ - :func:`get_shear_matrix2d`: the 2D shear from two angles, ``(B, 3, 3)``.+ - :func:`get_affine_matrix3d`: this shear composed with a rotation, a scale and a translation.+ - :func:`warp_perspective3d`: warp a volume by the returned matrix.Examples:>>> rng = torch.manual_seed(0)@@ -1409,9 +1593,6 @@ def get_shear_matrix3d([ 0.3022, -1.0729, 1.0000, 0.0000],[ 0.0000, 0.0000, 0.0000, 1.0000]]])- .. note::- This function is often used in conjunction with :func:`warp_perspective3d`.-"""sxy = torch.zeros(center.size(0), device=center.device, dtype=center.dtype) if sxy is None else sxysxz = torch.zeros(center.size(0), device=center.device, dtype=center.dtype) if sxz is None else sxz@@ -1471,17 +1652,24 @@ def warp_affine3d(padding_mode: str = "zeros",align_corners: bool = True,) -> torch.Tensor:- r"""Apply a projective transformation a to 3d torch.Tensor.+ """Warp a volume by a sourceādestination 3x4 affine matrix in voxel coordinates.++ ``out = warp_affine3d(src, M, dsize)`` takes volumes ``(B, C, D, H, W)`` and one ``(B, 3, 4)`` affine matrix per+ volume that maps **source** voxels ``(x, y, z)`` to **destination** voxels, and returns the ``(B, C, d, h, w)``+ destination volumes for ``dsize = (d, h, w)``; ``flags`` is the interpolation mode. Keep the default+ ``align_corners=True``: at ``False`` the matrix is normalized for corner-aligned voxels while ``grid_sample`` reads+ half-voxel ones, so a translation of ``t`` voxels moves the volume ``t * size / (size - 1)`` voxels along that+ axis (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For a ``(B, 4, 4)`` homography, use+ :func:`warp_perspective3d`; for images, :func:`warp_affine`; to keep the input size, :func:`affine3d`... warning::This API signature it is experimental and might suffer some changes in the future.Convention:- - input: ``(B, C, D, H, W)``; ``dsize`` is ``(d, h, w)``- - ``M`` is the sourceādestination **pixel** affine matrix ``(B, 3, 4)``- align_corners: ``True`` by default- padding_mode: ``'zeros'`` by default- - an empty batch returns shape :math:`(0, C, d, h, w)` with gradients connected to both inputs+ - ``M`` has the batch of ``src``: one matrix is not shared by a larger batch+ - an empty batch returns shape ``(0, C, d, h, w)`` with gradients connected to both inputseven when a source spatial dimension is zero- a zero output dimension returns an autograd-connected empty tensor;negative output dimensions raise ``ValueError``@@ -1489,7 +1677,7 @@ def warp_affine3d(Args:src : input torch.Tensor of shape ``(B, C, D, H, W)``.- M: projective transformation matrix of shape ``(B, 3, 4)``.+ M: affine transformation matrix of shape ``(B, 3, 4)``.dsize: size of the output image (depth, height, width).flags: interpolation mode to calculate output values``'bilinear'`` | ``'nearest'``.@@ -1498,11 +1686,17 @@ def warp_affine3d(align_corners : mode for grid_generation.Returns:- torch.Tensor: the warped 3d torch.tensor with shape :math:`(B, C, d, h, w)`, spatial sizes+ torch.Tensor: the warped 3d torch.tensor with shape ``(B, C, d, h, w)``, spatial sizesgiven by ``dsize``.- .. note::- This function is often used in conjunction with :func:`get_perspective_transform3d`.+ See Also:+ - :func:`warp_affine`: the 2D warp of images by ``(B, 2, 3)`` matrices.+ - :func:`warp_perspective3d`: a ``(B, 4, 4)`` homography; its default ``align_corners=False`` resamples even an+ identity.+ - :func:`affine3d`: keeps the input size and accepts an unbatched ``(C, D, H, W)`` volume.+ - :func:`get_projective_transform`: the ``(B, 3, 4)`` matrix of a rotation and a scale about a centre.+ - :func:`crop_by_transform_mat3d`: calls this warp with the top three rows of a ``(B, 3, 4)`` or ``(B, 4, 4)``+ matrix, and broadcasts a batch-1 matrix."""if len(src.shape) != 5:@@ -1542,13 +1736,16 @@ def warp_affine3d(def projection_from_Rt(rmat: torch.Tensor, tvec: torch.Tensor) -> torch.Tensor:- r"""Compute the projection matrix from Rotation and translation.+ """Stack rotations and translations into ``[R | t]`` matrices of shape ``(*, 3, 4)``.++ ``P = projection_from_Rt(rmat, tvec)`` concatenates rotations ``(*, 3, 3)`` and translations ``(*, 3, 1)`` with the+ same leading dimensions into the ``(*, 3, 4)`` matrices :math:`P = [R | t]`. It applies no intrinsics: for a camera+ projection matrix :math:`K [R | t]`, use :func:`~kornia.geometry.epipolar.projection_from_KRt`, and for a 4x4+ pose, :func:`~kornia.geometry.conversions.Rt_to_matrix4x4`... warning::This API signature it is experimental and might suffer some changes in the future.- Concatenates the batch of rotations and translations such that :math:`P = [R | t]`.-Convention:- returns the concatenation :math:`[R | t]` with shape ``(*, 3, 4)``@@ -1559,6 +1756,10 @@ def projection_from_Rt(rmat: torch.Tensor, tvec: torch.Tensor) -> torch.Tensor:Returns:the projection matrix with shape ``(*, 3, 4)``.+ See Also:+ - :func:`~kornia.geometry.epipolar.projection_from_KRt`: also multiplies by the camera intrinsics ``K``.+ - :func:`~kornia.geometry.conversions.Rt_to_matrix4x4`: the homogeneous ``(B, 4, 4)`` pose instead.+"""if not (len(rmat.shape) >= 2 and rmat.shape[-2:] == (3, 3)):raise AssertionError(rmat.shape)@@ -1569,13 +1770,19 @@ def projection_from_Rt(rmat: torch.Tensor, tvec: torch.Tensor) -> torch.Tensor:def get_projective_transform(center: torch.Tensor, angles: torch.Tensor, scales: torch.Tensor) -> torch.Tensor:- r"""Calculate the projection matrix for a 3D rotation.+ """Compute the 3x4 affine matrix of a 3D scaling and axis-angle rotation about a centre.++ ``P = get_projective_transform(center, angles, scales)`` takes ``(B, 3)`` centres ``(x, y, z)`` in pixels,+ axis-angle vectors ``(rx, ry, rz)`` in degrees and per-axis scale factors, and returns the ``(B, 3, 4)``+ sourceādestination pixel matrix that scales and then rotates about ``center``, as :func:`warp_affine3d` takes. The+ rotation is Rodrigues' formula on ``angles`` converted to radians and follows the right-hand rule, so a+ positive rotation about ``z`` turns clockwise on screen, the opposite of :func:`get_rotation_matrix2d`. The matrix+ is affine, not projective; to add a translation and shears, use :func:`get_affine_matrix3d`, whose angles turn+ the other way (`#4408 <https://github.com/kornia/kornia/issues/4408>`_)... warning::This API signature it is experimental and might suffer some changes in the future.- The function computes the projection matrix given the center and angles per axis.-Convention:- ``center`` is ``(x, y, z)`` in pixels, origin at the top-left of the first depthslice (``z = 0``)@@ -1594,8 +1801,12 @@ def get_projective_transform(center: torch.Tensor, angles: torch.Tensor, scales:Returns:the projection matrix of 3D rotation with shape ``(B, 3, 4)``.- .. note::- This function is often used in conjunction with :func:`warp_affine3d`.+ See Also:+ - :func:`get_rotation_matrix2d`: the 2D rotation and scale, counter-clockwise on screen.+ - :func:`get_affine_matrix3d`: adds a translation and shears, and negates the angles.+ - :func:`rotate3d`: rotate a volume about its centre by such an axis-angle vector.+ - :func:`~kornia.geometry.conversions.axis_angle_to_rotation_matrix`: the 3x3 rotation of an axis-angle vector+ in radians."""if not (len(center.shape) == 2 and center.shape[-1] == 3):@@ -1633,7 +1844,13 @@ def get_projective_transform(center: torch.Tensor, angles: torch.Tensor, scales:def get_perspective_transform3d(src: torch.Tensor, dst: torch.Tensor) -> torch.Tensor:- r"""Calculate a 3d perspective transform from five of eight pairs of corresponding box corners.+ r"""Compute the 3D homography that maps five source box corners exactly onto destination ones.++ ``M = get_perspective_transform3d(src, dst)`` takes two ``(B, 8, 3)`` sets of ``(x, y, z)`` box corners in+ :func:`~kornia.geometry.bbox.bbox_generator3d` order and returns the ``(B, 4, 4)`` homography, with+ ``M[3, 3] = 1``, that maps corners 0, 1, 2, 5 and 7 of ``src`` onto those of ``dst``: the sourceādestination voxel+ homography that :func:`warp_perspective3d` takes. The other three corners map onto theirs only when all eight pairs+ are related by one homography. For 2D points, use :func:`get_perspective_transform`.The function calculates the matrix of a perspective transform so that:@@ -1705,8 +1922,10 @@ def get_perspective_transform3d(src: torch.Tensor, dst: torch.Tensor) -> torch.TReturns:the perspective transformation with shape ``(B, 4, 4)``.- .. note::- This function is often used in conjunction with :func:`warp_perspective3d`.+ See Also:+ - :func:`get_perspective_transform`: the 2D homography from four point pairs.+ - :func:`warp_perspective3d`: warp a volume by the returned homography.+ - :func:`~kornia.geometry.bbox.bbox_generator3d`: box corners in the order this function reads them."""if not isinstance(src, (torch.Tensor)):@@ -1860,10 +2079,17 @@ def warp_perspective3d(border_mode: str = "zeros",align_corners: bool = False,) -> torch.Tensor:- r"""Warp a 3D (B, C, D, H, W) volume by a (B, 4, 4) sourceādestination homography in voxel coordinates.+ r"""Warp a volume by a sourceādestination 4x4 homography in voxel coordinates.- The function warp_perspective3d transforms the source volume using- the specified matrix:+ ``out = warp_perspective3d(src, M, dsize)`` takes volumes ``(B, C, D, H, W)`` and one ``(B, 4, 4)`` homography per+ volume that maps **source** voxels ``(x, y, z)`` to **destination** voxels, and returns the ``(B, C, d, h, w)``+ destination volumes for ``dsize = (d, h, w)``; ``flags`` is the interpolation mode and ``border_mode`` the padding+ mode. Pass ``align_corners=True``: at the default ``False`` the sampling grid is corner-aligned while+ ``grid_sample`` reads half-voxel coordinates, so even an identity ``M`` resamples the volume+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For images, use :func:`warp_perspective`; for a+ ``(B, 3, 4)`` affine matrix, :func:`warp_affine3d`.++ Each output voxel samples the source at:.. math::\text{dst}(x, y, z) = \text{src}\left( \pi\left( M^{-1} \cdot (x, y, z, 1)^{T} \right) \right)@@ -1871,8 +2097,8 @@ def warp_perspective3d(where :math:`\pi` divides by the fourth (homogeneous) coordinate.Convention:- - input: ``(B, C, D, H, W)``; ``dsize`` is ``(d, h, w)``- - ``M`` is the sourceādestination **pixel** homography ``(B, 4, 4)``+ - ``M`` is the sourceādestination **pixel** homography ``(B, 4, 4)``, with the batch of ``src``; an+ unbatched ``(4, 4)`` ``M`` is accepted for a single volume- align_corners: ``False`` by default (differs from the 2D :func:`warp_perspective`,whose default is ``True``)- border_mode: ``'zeros'`` by default@@ -1890,10 +2116,13 @@ def warp_perspective3d(align_corners: interpolation flag.Returns:- the warped input image :math:`(B, C, d, h, w)`, spatial sizes given by ``dsize``.+ the warped input image ``(B, C, d, h, w)``, spatial sizes given by ``dsize``.- .. note::- This function is often used in conjunction with :func:`get_perspective_transform3d`.+ See Also:+ - :func:`warp_perspective`: the 2D warp of images by ``(B, 3, 3)`` homographies.+ - :func:`warp_affine3d`: a ``(B, 3, 4)`` affine matrix, correct at its default ``align_corners=True``.+ - :func:`homography_warp3d`: a destinationāsource homography, in normalized coordinates by default.+ - :func:`get_perspective_transform3d`: the homography that maps box corners onto box corners."""if not isinstance(src, torch.Tensor):@@ -1937,23 +2166,28 @@ def homography_warp(normalized_coordinates: bool = True,normalized_homography: bool = True,) -> torch.Tensor:- r"""Warp an image by a destinationāsource homography in normalized ``[-1, 1]`` coordinates (the defaults).-- For a sourceādestination homography in pixel coordinates, as :class:`~kornia.geometry.ransac.RANSAC` and- :func:`~kornia.geometry.homography.find_homography_dlt` return it, use :func:`warp_perspective`. Passed here- unchanged, such a homography is read as a normalized destinationāsource one, which typically samples outside- the source, so the output is mostly or entirely zeros (with the default ``padding_mode="zeros"``).- ``normalized_homography=False`` instead hands it to :func:`warp_perspective` with this function's- ``align_corners=False`` default, which for a pixel homography samples the same positions as- ``align_corners=True`` (see ``align_corners`` below).-- See :class:`~kornia.geometry.transform.HomographyWarper` for details.+ """Warp an image by a destinationāsource homography in normalized ``[-1, 1]`` coordinates.++ ``out = homography_warp(patch_src, src_homo_dst, dsize)`` takes images ``(N, C, H, W)`` and one ``(N, 3, 3)``+ homography per image that maps **destination** points to **source** points in **normalized** :math:`[-1, 1]`+ coordinates, and returns the ``(N, C, h, w)`` destination images for ``dsize = (h, w)``. In normalized coordinates+ an identity maps the whole source onto the whole output, so a different ``dsize`` resizes the image. For a+ sourceādestination homography in **pixel** coordinates, as :func:`get_perspective_transform`,+ :func:`~kornia.geometry.homography.find_homography_dlt` and :class:`~kornia.geometry.ransac.RANSAC` return it, use+ :func:`warp_perspective`. Passed here unchanged, such a homography is read as a normalized destinationāsource one,+ which typically samples outside the source, so the output is mostly or entirely zeros (with the default+ ``padding_mode="zeros"``).++ ``normalized_coordinates=False`` reads the same destinationāsource homography in pixels, and+ ``normalized_homography=False`` instead hands a pixel sourceādestination homography to :func:`warp_perspective`+ with this function's ``align_corners=False`` default, which for a pixel homography samples the same positions as+ ``align_corners=True`` (see ``align_corners`` below), up to which pixel ``mode="nearest"`` takes for a sample+ exactly halfway between two pixels.Convention:See :doc:`Conventions & Pitfalls </get-started/conventions>` for homography direction, normalizedcoordinates and ``align_corners`` sampling rules.- - input: ``(N, C, H, W)``- ``src_homo_dst`` is the destinationāsource homography ``(N, 3, 3)`` when``normalized_homography=True`` (default), in normalized :math:`[-1, 1]` coordinates by default(``normalized_coordinates=True``). With ``normalized_coordinates=False`` it is in **pixel**@@ -1961,12 +2195,14 @@ def homography_warp(source pixel it samples, so an identity returns the top-left ``dsize`` window of the input- with ``normalized_homography=False`` it is consumed as the sourceādestination **pixel** homography,exactly like :func:`warp_perspective`, and ``normalized_coordinates`` is ignored+ - ``src_homo_dst`` has the batch of ``patch_src``: a ``(1, 3, 3)`` homography is not shared by a larger+ batch- with ``normalized_coordinates=False`` the pixel grid is warped and normalized in at least float32, alsounder a half default dtype and in a caller's autocast region, and then cast to the homography's dtype- - ``dsize`` is ``(h, w)``- align_corners: ``False`` by default (differs from :func:`warp_perspective`). Along a source axis longerthan one pixel, a pixel homography samples the same positions under both settings, which then differ- only in ``padding_mode="reflection"``+ only in ``padding_mode="reflection"`` and, with ``mode="nearest"``, in which pixel a sample exactly+ halfway between two pixels takes- negative output dimensions raise ``ValueError``Args:@@ -1985,7 +2221,16 @@ def homography_warp((``False``).Return:- Patch sampled at locations from source to destination.+ the warped images ``(N, C, h, w)``, sampled at the source locations of the destination pixels.++ See Also:+ - :func:`warp_perspective`: takes the sourceādestination homography in pixel coordinates.+ - :class:`HomographyWarper`: the module form, which can precompute the sampling grid for repeated warps.+ - :func:`homography_warp3d`: the same warp for ``(N, C, D, H, W)`` volumes by ``(N, 4, 4)`` homographies.+ - :func:`warp_grid`: the warped sampling grid alone, without sampling the image.+ - :func:`~kornia.geometry.conversions.normalize_homography`: convert a pixel sourceādestination homography to+ normalized coordinates, with this function's ``align_corners`` (it defaults to ``True`` there); invert the+ result for this function.Example:>>> input = torch.rand(1, 3, 32, 32)@@ -2060,12 +2305,17 @@ def homography_warp3d(align_corners: bool = False,normalized_coordinates: bool = True,) -> torch.Tensor:- r"""Warp a volume by a normalized destinationāsource 3D homography (default ``normalized_coordinates=True``).-- For a sourceādestination homography in voxel coordinates, use :func:`warp_perspective3d` with- ``align_corners=True``: at its default ``align_corners=False`` it resamples even an identity- (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). Alternatively pass the inverse of that homography- here with ``normalized_coordinates=False``. This function has no ``normalized_homography`` switch.+ """Warp a volume by a destinationāsource 3D homography in normalized coordinates.++ ``out = homography_warp3d(patch_src, src_homo_dst, dsize)`` takes volumes ``(N, C, D, H, W)`` and one+ ``(N, 4, 4)`` homography per volume that maps **destination** points ``(x, y, z)`` to **source** points in+ **normalized** :math:`[-1, 1]` coordinates, and returns the ``(N, C, d, h, w)`` destination volumes for+ ``dsize = (d, h, w)``. In normalized coordinates, pass ``align_corners=True``: the normalized grid is always+ corner-aligned, so at the default ``False`` even an identity resamples the volume+ (`#4503 <https://github.com/kornia/kornia/issues/4503>`_). For a sourceādestination homography in voxel+ coordinates, use :func:`warp_perspective3d` with ``align_corners=True``, which at its default ``False`` resamples+ even an identity for the same reason, or pass the inverse of that homography here with+ ``normalized_coordinates=False``. This function has no ``normalized_homography`` switch.Convention:- input: ``(N, C, D, H, W)``; ``dsize`` is the output size ``(d, h, w)``@@ -2087,7 +2337,8 @@ def homography_warp3d(the homography's dtype- align_corners: ``False`` by default. With ``normalized_coordinates=False`` and a source axislonger than one voxel, a voxel homography samples the same positions under both settings,- which then differ only in ``padding_mode="reflection"``+ which then differ only in ``padding_mode="reflection"`` and, with ``mode="nearest"``, in which voxel a+ sample exactly halfway between two voxels takes- padding_mode: ``'zeros'`` by default- negative output dimensions raise ``ValueError``@@ -2103,7 +2354,13 @@ def homography_warp3d((``True``) or in voxel coordinates (``False``).Return:- Patch sampled at locations from source to destination.+ the warped volumes ``(N, C, d, h, w)``, sampled at the source locations of the destination voxels.++ See Also:+ - :func:`homography_warp`: the 2D warp of images, which also has a ``normalized_homography`` switch.+ - :func:`warp_perspective3d`: takes the sourceādestination homography in voxel coordinates.+ - :func:`warp_grid3d`: the warped sampling grid alone, in ``(x, y, z)`` order.+ - :func:`~kornia.geometry.grid.create_meshgrid3d`: kornia's ``(d, x, y)`` grid, which this function reorders.Example:>>> input = torch.rand(1, 3, 8, 32, 32)diff --git a/kornia/geometry/transform/pyramid.py b/kornia/geometry/transform/pyramid.pyindex 8647e7c57..f39f9863c 100644--- a/kornia/geometry/transform/pyramid.py+++ b/kornia/geometry/transform/pyramid.py@@ -15,6 +15,28 @@# limitations under the License.#+"""Build Gaussian, Laplacian and scale-space pyramids, and blur and resize by one level.++:func:`pyrdown` and :func:`pyrup`, and the pyramids built from them, take channels-first images ``(B, C, H, W)``,+blur with the 5x5 binomial kernel (the outer product of ``[1, 4, 6, 4, 1] / 16`` with itself) and resize bilinearly:+``pyrdown`` blurs and then resizes to ``floor(side / factor)``, and ``pyrup`` resizes to twice the size and then+blurs. They default to ``align_corners=False`` and pad the blur by ``border_type``, ``'reflect'`` by default, under+which the blurred image needs at least 3 pixels per side. :class:`ScalePyramid` blurs with Gaussians of its own sigmas+and resizes with ``align_corners=True``.++Which function to use:++- One level down or up: :func:`pyrdown` or :class:`PyrDown`, :func:`pyrup` or :class:`PyrUp`.+- A list of Gaussian levels, each half the size of the previous: :func:`build_pyramid`. Band-pass levels that+ :func:`pyrup` and addition rebuild the image from: :func:`build_laplacian_pyramid`.+- Octaves of progressively blurred levels with their sigmas, for scale-space keypoint detection:+ :class:`ScalePyramid`.+- The integer helpers that :func:`build_laplacian_pyramid` pads with: :func:`is_powerof_two` and+ :func:`find_next_powerof_two`.+- To keep every second pixel of a blurred feature map in a network, use :func:`~kornia.filters.blur_pool2d`; to+ change the size without blurring, :func:`~kornia.geometry.transform.resize`.+"""+from __future__ import annotationsimport math@@ -59,7 +81,12 @@ def _get_pyramid_gaussian_kernel() -> torch.Tensor:class PyrDown(nn.Module):- r"""Blur a torch.Tensor and downsamples it.+ r"""Blur images with the 5x5 pyramid kernel and downsample them by a factor (module form).++ Build ``down = PyrDown()``, optionally with ``border_type``, ``align_corners`` and ``factor``, then+ ``out = down(input)`` blurs ``(B, C, H, W)`` images and resizes them to ``floor(H / factor)`` by+ ``floor(W / factor)``, as :func:`pyrdown` does. For the step up, use :class:`PyrUp`; for every level at once,+ :func:`build_pyramid`.Convention:- See the convention block of :func:`~kornia.geometry.transform.pyrdown`.@@ -76,7 +103,11 @@ class PyrDown(nn.Module):Shape:- Input: ``(B, C, H, W)``- - Output: ``(B, C, H / 2, W / 2)``+ - Output: ``(B, C, H // 2, W // 2)`` at the default ``factor=2.0``++ See Also:+ - :func:`pyrdown`: the one-call form.+ - :class:`PyrUp`: the module that goes one level up.Examples:>>> input = torch.rand(1, 2, 4, 4)@@ -111,7 +142,11 @@ class PyrDown(nn.Module):class PyrUp(nn.Module):- r"""Upsample a torch.Tensor and then blurs it.+ r"""Upsample images to twice their size and blur them with the 5x5 pyramid kernel (module form).++ Build ``up = PyrUp()``, optionally with ``border_type`` and ``align_corners``, then ``out = up(input)`` resizes+ ``(B, C, H, W)`` images to ``(B, C, H * 2, W * 2)`` and blurs them, as :func:`pyrup` does. For the step down, use+ :class:`PyrDown`.Convention:- See the convention block of :func:`~kornia.geometry.transform.pyrup`.@@ -129,6 +164,10 @@ class PyrUp(nn.Module):- Input: ``(B, C, H, W)``- Output: ``(B, C, H * 2, W * 2)``+ See Also:+ - :func:`pyrup`: the one-call form.+ - :class:`PyrDown`: the module that goes one level down.+Examples:>>> input = torch.rand(1, 2, 4, 4)>>> output = PyrUp()(input) # 1x2x8x8@@ -160,9 +199,16 @@ class PyrUp(nn.Module):class ScalePyramid(nn.Module):- r"""Create an scale pyramid of image, usually used for local feature detection.+ r"""Build octaves of progressively blurred images, with their sigmas, for scale-space detection.- Images are consequently smoothed with Gaussian blur and downscaled.+ Build ``sp = ScalePyramid(n_levels, init_sigma, min_size)``, then ``pyr, sigmas, pixel_dists = sp(input)`` takes+ ``(B, C, H, W)`` images and returns three lists with one entry per octave. ``pyr[o]`` stacks+ ``L = n_levels + extra_levels`` levels as ``(B, C, L, H_o, W_o)``, each blurred ``2 ** (1 / n_levels)`` times more+ than the previous, and ``sigmas[o]`` and ``pixel_dists[o]`` are ``(B, L)``. Each octave after the+ first starts from level ``n_levels`` of the previous one (for ``extra_levels >= 1``,+ `#6400 <https://github.com/kornia/kornia/issues/6400>`_), resized to half its size, and+ octaves are added while that halved size stays above ``min_size`` on its shorter side. It is the pyramid+ :class:`~kornia.feature.ScaleSpaceDetector` searches; for a plain list of halved images, use :func:`build_pyramid`.Convention:- input: ``(B, C, H, W)``@@ -181,19 +227,30 @@ class ScalePyramid(nn.Module):Args:n_levels: number of the levels in octave.init_sigma: initial blur level.- min_size: the minimum size of the octave in pixels.+ min_size: the minimum size of the octave in pixels: an octave after the first is built only when its shorter+ side is larger than ``min_size``.double_image: add 2x upscaled image as 1st level of pyramid. OpenCV SIFT does this.+ extra_levels: number of levels per octave beyond ``n_levels``; the default 3 is what DoG non-maximum+ suppression needs. Keep it at least 1: the next octave starts from level ``n_levels``, which+ ``extra_levels=0`` leaves out of the octave, and then starts from the least blurred level instead+ (`#6400 <https://github.com/kornia/kornia/issues/6400>`_).Returns:- 1st output: images- 2nd output: sigmas (coefficients for scale conversion)- 3rd output: pixelDists (nominal pixel spacing of each level relative to the input)+ 1st output: images, the levels of each octave.+ 2nd output: sigmas, the nominal blur of each level in the octave's own pixels.+ 3rd output: pixel_dists, the nominal pixel spacing of each level relative to the input.Shape:- Input: ``(B, C, H, W)``- - Output 1st: :math:`[(B, C, NL, H, W), (B, C, NL, H/2, W/2), ...]`- - Output 2nd: :math:`[(B, NL), (B, NL), (B, NL), ...]`- - Output 3rd: :math:`[(B, NL), (B, NL), (B, NL), ...]`+ - Output 1st: ``[(B, C, L, H, W), (B, C, L, H // 2, W // 2), ...]``, starting from ``(B, C, L, H * 2, W * 2)``+ with ``double_image=True``+ - Output 2nd: ``[(B, L), (B, L), (B, L), ...]``+ - Output 3rd: ``[(B, L), (B, L), (B, L), ...]``++ See Also:+ - :func:`build_pyramid`: a list of halved images, one blur per level, without sigmas.+ - :func:`build_laplacian_pyramid`: band-pass levels for reconstruction rather than detection.+ - :class:`~kornia.feature.ScaleSpaceDetector`: detects keypoints in this pyramid.Examples:>>> input = torch.rand(2, 4, 100, 100)@@ -415,7 +472,13 @@ class ScalePyramid(nn.Module):def pyrdown(input: torch.Tensor, border_type: str = "reflect", align_corners: bool = False, factor: float = 2.0) -> torch.Tensor:- r"""Blur a torch.Tensor and downsamples it.+ r"""Blur images with the 5x5 pyramid kernel and downsample them by a factor, one pyramid level down.++ ``out = pyrdown(input)`` takes images ``(B, C, H, W)``, blurs them with the 5x5 binomial kernel and resizes them+ bilinearly to ``(B, C, floor(H / factor), floor(W / factor))``, half the size at the default ``factor=2.0``. It+ resamples bilinearly where OpenCV's ``pyrDown`` keeps every second pixel, and rounds an odd side down where OpenCV+ rounds it up. For the step up, use :func:`pyrup`; for every level at once, :func:`build_pyramid`; to keep+ every second pixel of a blurred map, :func:`~kornia.filters.blur_pool2d`... image:: _static/img/pyrdown.png@@ -426,7 +489,7 @@ def pyrdown(note OpenCV's ``pyrDown`` rounds up instead)- Dynamo ONNX uses float32 reciprocal-scale arithmetic for dynamic sides, which can differ from eager by onepixel for some fractional factors (for example, side 33 and factor 1.1)- - border_type: ``'reflect'`` by default (blur before downsample)+ - border_type: ``'reflect'`` by default (blur before downsample), which needs ``H`` and ``W`` of at least 3- align_corners: ``False`` by defaultArgs:@@ -440,6 +503,13 @@ def pyrdown(Return:the downsampled torch.Tensor.+ See Also:+ - :class:`PyrDown`: the module form.+ - :func:`pyrup`: one level up, to twice the size.+ - :func:`build_pyramid`: the list of levels from repeated ``pyrdown`` calls.+ - :func:`~kornia.filters.blur_pool2d`: a zero-padded binomial blur sampled at a stride, from pixel 0;+ ``blur_pool2d(x, 5, 2)`` blurs with this kernel and returns ``ceil(H / 2)`` rows.+Examples:>>> input = torch.arange(16, dtype=torch.float32).reshape(1, 1, 4, 4)>>> pyrdown(input, align_corners=True)@@ -471,7 +541,12 @@ def pyrdown(def pyrup(input: torch.Tensor, border_type: str = "reflect", align_corners: bool = False) -> torch.Tensor:- r"""Upsample a torch.Tensor and then blurs it.+ r"""Upsample images to twice their size and blur them with the 5x5 pyramid kernel, one pyramid level up.++ ``out = pyrup(input)`` takes images ``(B, C, H, W)``, resizes them bilinearly to ``(B, C, H * 2, W * 2)`` and then+ blurs them with the 5x5 binomial kernel; it has no ``factor``. With ``pyrdown`` it builds the levels of+ :func:`build_laplacian_pyramid`. For the step down, use :func:`pyrdown`; to change the size without blurring,+ :func:`~kornia.geometry.transform.resize`... image:: _static/img/pyrup.png@@ -491,6 +566,11 @@ def pyrup(input: torch.Tensor, border_type: str = "reflect", align_corners: boolReturn:the upsampled torch.Tensor.+ See Also:+ - :class:`PyrUp`: the module form.+ - :func:`pyrdown`: one level down, by any factor.+ - :func:`build_laplacian_pyramid`: subtracts ``pyrup`` of each Gaussian level from the next finer one.+Examples:>>> input = torch.arange(4, dtype=torch.float32).reshape(1, 1, 2, 2)>>> pyrup(input, align_corners=True)@@ -518,12 +598,15 @@ def pyrup(input: torch.Tensor, border_type: str = "reflect", align_corners: booldef build_pyramid(input: torch.Tensor, max_level: int, border_type: str = "reflect", align_corners: bool = False) -> list[torch.Tensor]:- r"""Construct the Gaussian pyramid for a torch.Tensor image.+ r"""Build a Gaussian pyramid: the image and its successively halved levels, as a list.- .. image:: _static/img/build_pyramid.png+ ``levels = build_pyramid(input, max_level)`` takes images ``(B, C, H, W)`` and returns a list of ``max_level``+ tensors: the input itself, then each previous level through :func:`pyrdown`, ``(B, C, H // 2, W // 2)`` and so on.+ With ``max_level > 1`` and the default ``'reflect'`` border every level but the last needs 3 pixels per side, so+ ``H`` and ``W`` must be at least ``3 * 2 ** (max_level - 2)``. For band-pass levels, use+ :func:`build_laplacian_pyramid`; for octaves of several blurs each, with their sigmas, :class:`ScalePyramid`.- The function constructs a vector of images and builds the Gaussian pyramid- by recursively applying pyrDown to the previously built pyramid layers.+ .. image:: _static/img/build_pyramid.pngConvention:- input: ``(B, C, H, W)``@@ -543,7 +626,12 @@ def build_pyramid(Shape:- Input: ``(B, C, H, W)``- - Output :math:`[(B, C, H, W), (B, C, H/2, W/2), ...]`+ - Output ``[(B, C, H, W), (B, C, H // 2, W // 2), ...]``++ See Also:+ - :func:`build_laplacian_pyramid`: the differences of these levels, ending with the coarsest one.+ - :class:`ScalePyramid`: octaves of several blurred levels each, with their sigmas, for keypoint detection.+ - :func:`pyrdown`: one level of this pyramid."""KORNIA_CHECK_SHAPE(input, ["B", "C", "H", "W"])@@ -566,7 +654,10 @@ def build_pyramid(def is_powerof_two(x: int) -> bool:- r"""Check whether a number is a power of two.+ r"""Check whether an integer is a positive power of two.++ ``is_powerof_two(x)`` returns ``True`` for 1, 2, 4, 8, ... and ``False`` for every other integer, ``0`` and+ negative numbers included. :func:`build_laplacian_pyramid` uses it to decide whether to pad an image.Args:x: the integer to check.@@ -574,6 +665,9 @@ def is_powerof_two(x: int) -> bool:Returns:``True`` if ``x`` is a positive power of two, ``False`` otherwise (including for ``0``).+ See Also:+ - :func:`find_next_powerof_two`: the smallest power of two not below ``x``.+Example:>>> is_powerof_two(64)True@@ -585,7 +679,10 @@ def is_powerof_two(x: int) -> bool:def find_next_powerof_two(x: int) -> int:- r"""Find the smallest power of two that is greater than or equal to a number.+ r"""Find the smallest power of two that is greater than or equal to a positive integer.++ ``find_next_powerof_two(x)`` returns ``x`` itself when it is a power of two, and the next one above otherwise.+ It is defined for positive integers only. :func:`build_laplacian_pyramid` pads each side of an image to it.Args:x: a positive integer.@@ -593,6 +690,9 @@ def find_next_powerof_two(x: int) -> int:Returns:the smallest power of two not below ``x``.+ See Also:+ - :func:`is_powerof_two`: whether ``x`` is already a power of two.+Example:>>> find_next_powerof_two(100)128@@ -606,11 +706,17 @@ def find_next_powerof_two(x: int) -> int:def build_laplacian_pyramid(input: torch.Tensor, max_level: int, border_type: str = "reflect", align_corners: bool = False) -> list[torch.Tensor]:- r"""Construct the Laplacian pyramid for a torch.Tensor image.-- The function constructs a vector of images and builds the Laplacian pyramid- by recursively computing the difference after applying- pyrUp to the adjacent layer in its Gaussian pyramid.+ r"""Build a Laplacian pyramid: band-pass differences of Gaussian levels, ending with the coarsest level.++ ``levels = build_laplacian_pyramid(input, max_level)`` takes images ``(B, C, H, W)`` and returns a list of+ ``max_level`` tensors: level ``i`` is Gaussian level ``i`` of :func:`build_pyramid` minus :func:`pyrup` of Gaussian+ level ``i + 1``, and the last is the coarsest Gaussian level. Starting from ``x = levels[-1]``,+ ``x = pyrup(x, border_type, align_corners) + levels[i]`` for ``i`` from ``max_level - 2`` down to 0, with the+ ``border_type`` and ``align_corners`` the pyramid was built with, rebuilds the input up to float rounding. The+ input is first reflect-padded at the right and bottom to the next power of two per side when neither side is a+ power of two or a side does not halve evenly ``max_level - 1`` times (see Convention); every level is then shaped+ from the padded size, and the rebuilt image is the padded one. For the Gaussian levels themselves, use+ :func:`build_pyramid`.See :cite:`burt1987laplacian` for more details.@@ -642,10 +748,15 @@ def build_laplacian_pyramid(align_corners: interpolation flag.Return:- Output: :math:`[(B, C, H', W'), (B, C, H'/2, W'/2), ...]`, where- :math:`(H', W')` equals the input ``(H, W)`` unless padding was+ Output: ``[(B, C, H', W'), (B, C, H' / 2, W' / 2), ...]``, where+ ``(H', W')`` equals the input ``(H, W)`` unless padding wasapplied (see Convention above), in which case it is the padded size.+ See Also:+ - :func:`build_pyramid`: the Gaussian levels these are differences of.+ - :func:`pyrup`: the upsampling step between levels, also the one that rebuilds the image.+ - :class:`ScalePyramid`: octaves of blurred levels for keypoint detection, with their sigmas.+"""KORNIA_CHECK_SHAPE(input, ["B", "C", "H", "W"])KORNIA_CHECK(diff --git a/kornia/geometry/transform/thin_plate_spline.py b/kornia/geometry/transform/thin_plate_spline.pyindex 8122597a4..65494f13c 100644--- a/kornia/geometry/transform/thin_plate_spline.py+++ b/kornia/geometry/transform/thin_plate_spline.py@@ -15,6 +15,28 @@# limitations under the License.#+r"""Fit a thin-plate spline to pairs of 2D points, and warp points or images with it.++Points are ``(B, N, 2)`` tensors of ``(x, y)`` coordinates. :func:`get_tps_transform` and :func:`warp_points_tps`+take them in whatever frame the caller uses and normalize nothing; :func:`warp_image_tps` needs them in the normalized+frame of ``grid_sample``. :func:`get_tps_transform` returns kernel weights ``(B, N, 2)`` and affine weights+``(B, 3, 2)``, with the x and y transforms as columns, for radial kernels centred on its **second** argument, which the+warps take as ``kernel_centers``. A warp then computes ``f(p) = a_0 + p @ A + sum_i w_i U(|p - c_i|)`` with+:math:`U(r) = r^2 \log r`, ``[a_0; A]`` the affine weights and ``c_i`` the kernel centres. Bookstein's thin-plate+spline centres them on the first argument instead, so these warps differ from it between and away from the landmarks+(`#6404 <https://github.com/kornia/kornia/issues/6404>`_).++Which function to use:++- Fit the spline that maps source points onto target points: :func:`get_tps_transform`.+- Move points with it: :func:`warp_points_tps`. Warp an image with it: :func:`warp_image_tps`, which samples the+ input at the spline of each output pixel, so fit it from target to source, in normalized coordinates.+- For one global matrix, use :func:`~kornia.geometry.transform.warp_perspective` or+ :func:`~kornia.geometry.transform.warp_affine`; for random spline warps in an augmentation pipeline,+ :class:`~kornia.augmentation.RandomThinPlateSpline`; for a smoothed random displacement field,+ :func:`~kornia.geometry.transform.elastic_transform2d`.+"""+from __future__ import annotationsimport torch@@ -56,16 +78,27 @@ def _kernel_distance(squared_distances: torch.Tensor) -> torch.Tensor:def get_tps_transform(points_src: torch.Tensor, points_dst: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:- r"""Compute the TPS transform parameters that warp source points to target points.+ r"""Fit a thin-plate-spline warp that maps source points exactly onto target points.- The input to this function is a torch.Tensor of :math:`(x, y)` source points ``(B, N, 2)`` and a corresponding- torch.Tensor of target :math:`(x, y)` points ``(B, N, 2)``.+ ``kernel_weights, affine_weights = get_tps_transform(points_src, points_dst)`` takes ``(B, N, 2)`` point pairs and+ returns the ``(B, N, 2)`` kernel weights and ``(B, 3, 2)`` affine weights of the spline whose kernels are centred on+ ``points_dst``: ``warp_points_tps(points_src, points_dst, kernel_weights, affine_weights)`` returns ``points_dst``.+ Unlike Bookstein's thin-plate spline, whose kernels are centred on the points where it interpolates+ (``points_src`` here), the kernels sit on ``points_dst``, so the warp differs between and away from the landmarks+ (`#6404 <https://github.com/kornia/kornia/issues/6404>`_). To warp an image, fit the reverse mapping and use+ :func:`warp_image_tps` (see Convention); for a homography from four point pairs, use+ :func:`get_perspective_transform`.Convention:- ``points_src``/``points_dst``: ``(B, N, 2)`` in ``(x, y)`` order, in whatevercoordinate frame the caller supplies; no normalization is applied- returns kernel weights ``(B, N, 2)`` and affine weights ``(B, 3, 2)``; the- identity mapping gives zero kernel weights and an identity affine up to solver round-off+ identity mapping gives zero kernel weights and an identity affine up to solver round-off, and any pairs+ related by an affine map give zero kernel weights, except a constant map of more than three points, whose+ equal ``points_dst`` make the system singular+ - zero or one point makes the linear system singular and ``torch.linalg.solve`` raises; two points, like any+ set of points all on one line, make it singular too, but round-off can hide that, and the solve then+ returns huge affine weights that throw points off that line far away and may miss the landmarks themselves- the tensor passed as the **second** argument is the ``kernel_centers`` that the warpfunction must receive with these weights. :func:`warp_points_tps` is composed in thenatural ``(points_src, points_dst)`` order; :func:`warp_image_tps` samples from output@@ -83,16 +116,19 @@ def get_tps_transform(points_src: torch.Tensor, points_dst: torch.Tensor) -> tupReturns:``(B, N, 2)`` torch.Tensor of kernel weights and ``(B, 3, 2)``torch.Tensor of affine weights. The last dimension contains the x-transform and y-transform weights- as separate columns.+ as separate columns. Row 0 of the affine weights is the translation and rows 1 and 2 the matrix ``A``+ that multiplies a point as ``p @ A``.++ See Also:+ - :func:`warp_points_tps`: applies the fitted spline to points, with ``points_dst`` as ``kernel_centers``.+ - :func:`warp_image_tps`: applies a spline fitted from target to source to an image.+ - :func:`get_perspective_transform`: the ``(B, 3, 3)`` homography of four point pairs, one global matrix.Example:>>> points_src = torch.rand(1, 5, 2)>>> points_dst = torch.rand(1, 5, 2)>>> kernel_weights, affine_weights = get_tps_transform(points_src, points_dst)- .. note::- This function is often used in conjunction with :func:`warp_points_tps`, :func:`warp_image_tps`.-"""if not isinstance(points_src, torch.Tensor):raise TypeError(f"Input points_src is not torch.Tensor. Got {type(points_src)}")@@ -133,13 +169,15 @@ def get_tps_transform(points_src: torch.Tensor, points_dst: torch.Tensor) -> tupdef warp_points_tps(points_src: torch.Tensor, kernel_centers: torch.Tensor, kernel_weights: torch.Tensor, affine_weights: torch.Tensor) -> torch.Tensor:- r"""Warp a torch.Tensor of coordinate points using the thin plate spline defined by arguments.+ r"""Move 2D points by a thin-plate spline given as kernel centres and weights.- The source points should be a ``(B, N, 2)`` torch.Tensor of :math:`(x, y)` coordinates. The kernel centers are- a ``(B, K, 2)`` torch.Tensor of :math:`(x, y)` coordinates. The kernel weights are a ``(B, K, 2)``- torch.Tensor, and the affine weights are a ``(B, 3, 2)`` torch.Tensor. For the weight tensors,- torch.Tensor[..., 0] contains the weights for the x-transform and torch.Tensor[..., 1] the weights- for the y-transform.+ ``warped = warp_points_tps(points_src, kernel_centers, kernel_weights, affine_weights)`` takes ``(B, N, 2)`` points+ and the ``(B, K, 2)`` kernel centres, ``(B, K, 2)`` kernel weights and ``(B, 3, 2)`` affine weights of a spline, and+ returns the ``(B, N, 2)`` warped points in the same frame, unclamped. With the weights of+ ``get_tps_transform(points_src, points_dst)``, pass ``points_dst`` as ``kernel_centers``: that function centres the+ kernels there, unlike Bookstein's thin-plate spline (`#6404 <https://github.com/kornia/kornia/issues/6404>`_). In+ the weight tensors, ``[..., 0]`` holds the weights of the x transform and ``[..., 1]`` those of the y transform. To+ warp an image, use :func:`warp_image_tps`.Convention:- points: ``(B, N, 2)`` (or ``(B, K, 2)`` for kernel centers) in ``(x, y)``@@ -156,6 +194,11 @@ def warp_points_tps(Returns:The ``(B, N, 2)`` torch.Tensor of warped source points, from applying the TPS transform.+ See Also:+ - :func:`get_tps_transform`: fits the weights from point pairs.+ - :func:`warp_image_tps`: applies a spline to every pixel of an image, in normalized coordinates.+ - :func:`~kornia.geometry.linalg.transform_points`: moves points by one ``(B, 3, 3)`` matrix instead.+Example:>>> points_src = torch.rand(1, 5, 2)>>> points_dst = torch.rand(1, 5, 2)@@ -163,9 +206,6 @@ def warp_points_tps(>>> warped = warp_points_tps(points_src, points_dst, kernel_weights, affine_weights)>>> warped_correct = torch.allclose(warped, points_dst)- .. note::- This function is often used in conjunction with :func:`get_tps_transform`.-"""if not isinstance(points_src, torch.Tensor):raise TypeError(f"Input points_src is not torch.Tensor. Got {type(points_src)}")@@ -212,17 +252,21 @@ def warp_image_tps(align_corners: bool = False,padding_mode: str = "zeros",) -> torch.Tensor:- r"""Warp an image torch.Tensor according to the thin plate spline transform defined by arguments.+ r"""Warp images by a thin-plate spline that maps each output pixel to the input pixel it samples.++ ``out = warp_image_tps(image, kernel_centers, kernel_weights, affine_weights)`` takes images ``(B, C, H, W)`` and a+ spline as in :func:`warp_points_tps` (one per image, or a batch of 1 for all), applies the spline to the normalized+ coordinates of every output pixel and returns the ``(B, C, H, W)`` images bilinearly sampled there. The spline+ therefore maps output space to input space: to move control points ``points_src`` of the input onto+ ``points_dst``, fit ``get_tps_transform(points_dst, points_src)`` and pass ``points_src`` as ``kernel_centers``,+ with both point sets in normalized coordinates. The kernels then sit on ``points_src``, whereas Bookstein's+ thin-plate spline would centre them on ``points_dst``, the output points where it interpolates+ (`#6404 <https://github.com/kornia/kornia/issues/6404>`_). For one global matrix, use+ :func:`~kornia.geometry.transform.warp_perspective`; for random spline warps in an augmentation pipeline,+ :class:`~kornia.augmentation.RandomThinPlateSpline`... image:: _static/img/warp_image_tps.png- The transform is applied to each pixel coordinate in the output image to obtain a point in the input- image for interpolation of the output pixel. So the TPS parameters should correspond to a warp from- output space to input space.-- The input `image` is a ``(B, C, H, W)`` torch.Tensor. The kernel centers, kernel weight and affine weights- are the same as in `warp_points_tps`.-Convention:- image: ``(B, C, H, W)``; the weights come from :func:`get_tps_transform` called**reversed**, ``get_tps_transform(points_dst, points_src)``, and ``kernel_centers`` is@@ -245,6 +289,13 @@ def warp_image_tps(Returns:warped image torch.Tensor ``(B, C, H, W)``.+ See Also:+ - :func:`get_tps_transform`: fits the spline; call it with the target points first for this warp.+ - :func:`warp_points_tps`: applies the same spline to points.+ - :class:`~kornia.augmentation.RandomThinPlateSpline`: random spline warps, in an augmentation pipeline.+ - :func:`~kornia.geometry.transform.elastic_transform2d`: a smoothed random displacement field instead of+ control points.+Example:>>> points_src = torch.rand(1, 5, 2)>>> points_dst = torch.rand(1, 5, 2)@@ -253,9 +304,6 @@ def warp_image_tps(>>> kernel_weights, affine_weights = get_tps_transform(points_dst, points_src)>>> warped_image = warp_image_tps(image, points_src, kernel_weights, affine_weights)- .. note::- This function is often used in conjunction with :func:`get_tps_transform`.-"""if not isinstance(image, torch.Tensor):raise TypeError(f"Input image is not torch.Tensor. Got {type(image)}")diff --git a/testing/docstring_rollout.py b/testing/docstring_rollout.pyindex 3af7ae63b..057289874 100644--- a/testing/docstring_rollout.py+++ b/testing/docstring_rollout.py@@ -52,10 +52,37 @@ EXCLUDED_PACKAGES: dict[str, str] = {}"""Top-level packages left out of the rollout, with the reason."""-CATALOG_NAMESPACES: tuple[str, ...] = ("kornia.geometry.depth",)+CATALOG_NAMESPACES: tuple[str, ...] = (+ "kornia.geometry.conversions",+ "kornia.geometry.depth",+ "kornia.geometry.epipolar",+ "kornia.geometry.homography",+ "kornia.geometry.pose",+ "kornia.geometry.ransac",+ "kornia.geometry.solvers",+ "kornia.geometry.transform.affwarp",+ "kornia.geometry.transform.crop2d",+ "kornia.geometry.transform.crop3d",+ "kornia.geometry.transform.elastic_transform",+ "kornia.geometry.transform.flips",+ "kornia.geometry.transform.homography_warper",+ "kornia.geometry.transform.image_registrator",+ "kornia.geometry.transform.imgwarp",+ "kornia.geometry.transform.pyramid",+ "kornia.geometry.transform.thin_plate_spline",+)"""Namespaces whose module docstring must name every export in ``__all__``."""-DOCSTRING_MODULES: tuple[str, ...] = ("kornia.geometry.depth",)+DOCSTRING_MODULES: tuple[str, ...] = (+ "kornia.geometry.conversions",+ "kornia.geometry.depth",+ "kornia.geometry.epipolar",+ "kornia.geometry.homography",+ "kornia.geometry.pose",+ "kornia.geometry.ransac",+ "kornia.geometry.solvers",+ "kornia.geometry.transform",+)"""Prefixes of the defining modules whose public objects carry the converted docstring style."""SEE_ALSO_EXCEPTIONS: dict[str, str] = {}diff --git a/testing/docstring_style.py b/testing/docstring_style.pyindex d4088f407..e00250390 100644--- a/testing/docstring_style.py+++ b/testing/docstring_style.py@@ -31,6 +31,7 @@ import inspectimport ioimport reimport sys+import textwrapimport tokenizeimport warningsimport zlib@@ -100,13 +101,8 @@ def _is_kornia_function_or_class(obj: object) -> bool:@cache-def public_objects() -> dict[int, tuple[object, str]]:- """Map ``id(obj)`` to ``(obj, "module.qualname")`` for every exported kornia function and class with a docstring.-- Covers every name in the ``__all__`` of a non-private kornia module that is a function or class defined in- kornia and has a docstring of its own, deduplicated by object identity. Names starting with an underscore are- out of scope. The dict is cached and shared between callers, who must not mutate it.- """+def _exported_objects() -> tuple[tuple[object, str], ...]:+ """List ``(obj, "module.qualname")`` for every exported kornia function and class, deduplicated by identity."""collected: dict[int, tuple[object, str]] = {}for module_name in modules_declaring_all():module = _import_exported(module_name)@@ -117,9 +113,30 @@ def public_objects() -> dict[int, tuple[object, str]]:warnings.simplefilter("ignore")exported = [getattr(module, name, None) for name in module.__all__ if not name.startswith("_")]for obj in exported:- if _is_kornia_function_or_class(obj) and own_docstring(obj):- collected[id(obj)] = (obj, f"{obj.__module__}.{obj.__qualname__}")- return collected+ if _is_kornia_function_or_class(obj):+ collected.setdefault(id(obj), (obj, f"{obj.__module__}.{obj.__qualname__}"))+ return tuple(collected.values())+++@cache+def public_objects() -> dict[int, tuple[object, str]]:+ """Map ``id(obj)`` to ``(obj, "module.qualname")`` for every exported kornia function and class with a docstring.++ Covers every name in the ``__all__`` of a non-private kornia module that is a function or class defined in+ kornia and has a docstring of its own, deduplicated by object identity. Names starting with an underscore are+ out of scope. The dict is cached and shared between callers, who must not mutate it.+ """+ return {id(obj): (obj, qualified) for obj, qualified in _exported_objects() if own_docstring(obj)}+++@cache+def undocumented_public_objects() -> dict[int, tuple[object, str]]:+ """Map ``id(obj)`` to ``(obj, "module.qualname")`` for every exported kornia function and class without a docstring.++ The complement of ``public_objects()`` over the same walk: ``public_objects()`` cannot see these, so a converted+ module could otherwise keep one forever. The dict is cached and shared between callers, who must not mutate it.+ """+ return {id(obj): (obj, qualified) for obj, qualified in _exported_objects() if not own_docstring(obj)}def covered(module_name: str, prefixes: Iterable[str]) -> bool:@@ -160,6 +177,153 @@ _PY_ROLES = "func|class|meth|attr|data|const|exc|obj"_XREF_ROLE = re.compile(rf":(?:py:)?({_PY_ROLES}):`~?([^`]+)`")+_STATUS_TITLE = re.compile(rf"(?:Alias of|Deprecated: use) :(?:py:)?(?:{_PY_ROLES}):`~?([^`\s]+)`\.")+++def status_target(doc: str) -> str | None:+ """Return the target of a status title line, or ``None`` when ``doc`` has another title.++ The status titles are ``Alias of :role:`target`.`` and ``Deprecated: use :role:`target`.``. Only the first line+ counts: it must read exactly ``Alias of`` or ``Deprecated: use``, one Python-domain role with an optional ``py:``+ prefix and an optional leading ``~``, and a closing period. The returned target has no ``~``.+ """+ match = _STATUS_TITLE.fullmatch(doc.strip().split("\n", 1)[0].strip())+ return match.group(1) if match else None+++def is_deprecated_wrapper(obj: object) -> bool:+ """Return whether ``obj`` is a function or class that ``kornia.core._compat.deprecated`` wraps.++ The decorator replaces a function by a wrapper, and the ``__init__`` of a class, with code of its own module.+ """+ import kornia.core._compat as compat++ code = getattr(obj.__init__ if isinstance(obj, type) else obj, "__code__", None)+ return code is not None and Path(code.co_filename).resolve() == Path(compat.__file__).resolve()+++def _definition_body(obj: object) -> tuple[ast.FunctionDef | ast.ClassDef, list[ast.stmt]]:+ """Return the AST definition of ``obj`` and its body without a leading docstring; ``OSError`` without source."""+ tree = ast.parse(textwrap.dedent(inspect.getsource(obj)))+ node = tree.body[0]+ if not isinstance(node, (ast.FunctionDef, ast.ClassDef)):+ raise OSError(f"no plain function or class definition of {obj!r} in its source")+ body = node.body+ if body and isinstance(body[0], ast.Expr) and isinstance(body[0].value, ast.Constant):+ body = body[1:]+ return node, body+++def _resolve_name(fn: object, expr: ast.expr) -> object | None:+ """Return the object that the name or dotted attribute ``expr`` denotes inside function ``fn``, or ``None``."""+ if isinstance(expr, ast.Attribute):+ owner = _resolve_name(fn, expr.value)+ return getattr(owner, expr.attr, None) if owner is not None else None+ if not isinstance(expr, ast.Name):+ return None+ names = inspect.getclosurevars(fn)+ for scope in (names.nonlocals, names.globals, names.builtins):+ if expr.id in scope:+ return scope[expr.id]+ return None+++def _forwards_own_arguments(args: ast.arguments, call: ast.Call) -> bool:+ """Whether ``call`` passes every parameter of ``args`` exactly once, unchanged and in place, and nothing else.++ Positional arguments must be the leading positional parameters in their order; ``*args`` may follow only once+ all of them are passed. Every other parameter goes by keyword under its own name, and ``**kwargs`` as such.+ """+ order = [arg.arg for arg in [*args.posonlyargs, *args.args]]+ by_keyword = {arg.arg for arg in [*args.args, *args.kwonlyargs]}+ passed: list[str] = []+ for value in call.args:+ if isinstance(value, ast.Starred):+ vararg = args.vararg.arg if args.vararg is not None else None+ if passed != order or not isinstance(value.value, ast.Name) or value.value.id != vararg:+ return False+ passed.append(value.value.id)+ elif isinstance(value, ast.Name) and len(passed) < len(order) and value.id == order[len(passed)]:+ passed.append(value.id)+ else:+ return False+ for keyword in call.keywords:+ if not isinstance(keyword.value, ast.Name):+ return False+ if keyword.arg is None:+ if args.kwarg is None or keyword.value.id != args.kwarg.arg:+ return False+ elif keyword.arg not in by_keyword or keyword.value.id != keyword.arg:+ return False+ passed.append(keyword.value.id)+ expected = [arg.arg for arg in [*args.posonlyargs, *args.args, *args.kwonlyargs]]+ expected += [extra.arg for extra in (args.vararg, args.kwarg) if extra is not None]+ return sorted(passed) == sorted(expected)+++def _keeps_target_defaults(obj: object, target: object, call: ast.Call) -> bool:+ """Whether every parameter ``obj`` gives a default reaches a ``target`` parameter with the same default.++ ``call`` is the forwarding call that :func:`_forwards_own_arguments` accepted, so its positional values map onto+ the leading positional parameters of ``target``, whatever their names, and its keywords onto the parameters of+ the same name. Defaults compare by ``repr``. A parameter without a default may meet one with a default: every+ call the alias accepts then behaves as the target. A default that differs, or that meets a required parameter,+ ``*args`` or ``**kwargs``, changes what a call relying on it returns.+ """+ try:+ own = inspect.signature(obj).parameters+ theirs = inspect.signature(target).parameters+ except (TypeError, ValueError):+ return False+ leading = [p for p in theirs.values() if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD)]+ reached: dict[str, inspect.Parameter | None] = {}+ for index, value in enumerate(call.args):+ if isinstance(value, ast.Name):+ reached[value.id] = leading[index] if index < len(leading) else None+ for keyword in call.keywords:+ if keyword.arg is not None and isinstance(keyword.value, ast.Name):+ parameter = theirs.get(keyword.arg)+ if parameter is not None and parameter.kind == parameter.VAR_KEYWORD:+ parameter = None+ reached[keyword.value.id] = parameter+ for name, parameter in own.items():+ if parameter.default is inspect.Parameter.empty:+ continue+ other = reached.get(name)+ if other is None or other.default is inspect.Parameter.empty or repr(other.default) != repr(parameter.default):+ return False+ return True+++def forwards_to(obj: object, target: object) -> bool:+ """Return whether ``obj`` only forwards to ``target``, which is what an ``Alias of`` title line promises.++ A function forwards when its body, docstring aside, is a single ``return target(...)`` that passes each of its+ own parameters exactly once and unchanged, the leading ones by position in their order and the rest as+ ``name=name``, with ``*args`` and ``**kwargs`` passed on as such, and nothing else, and when every default it+ declares is the default of the ``target`` parameter it reaches. A class forwards when ``target`` is its only+ base and its body, docstring aside, is empty or ``pass``. Neither may be decorated, since a decorator can add+ behaviour; an async function, or an object without Python source, does not forward either.+ """+ try:+ node, body = _definition_body(obj)+ except (OSError, TypeError, SyntaxError):+ return False+ if node.decorator_list:+ return False+ if isinstance(node, ast.ClassDef):+ bare = all(isinstance(statement, ast.Pass) for statement in body)+ return isinstance(obj, type) and obj.__bases__ == (target,) and not node.keywords and bare+ if len(body) != 1 or not isinstance(body[0], ast.Return) or not isinstance(body[0].value, ast.Call):+ return False+ call = body[0].value+ return (+ _resolve_name(obj, call.func) is target+ and _forwards_own_arguments(node.args, call)+ and _keeps_target_defaults(obj, target, call)+ )++def docstring_roles(doc: str) -> list[tuple[str, str]]:"""Return ``(role, target)`` for every Python-domain object role in ``doc``, without a leading ``~``.diff --git a/tests/geometry/test_conversions.py b/tests/geometry/test_conversions.pyindex 18e7e1bf4..93525362e 100644--- a/tests/geometry/test_conversions.py+++ b/tests/geometry/test_conversions.py@@ -906,8 +906,9 @@ class TestQuaternionToAngleAxis(BaseTester):# property of atan2 plus the 2*theta/||v|| factor, not of the particular numbers (verified# bit-identical in 500/500 random float64 quaternions, seeded torch.Generator(7), for both# factors -- a non-power-of-two factor such as 3 is invariant to rounding only, 82/500).- # Contrast quaternion_exp_to_log, which does NOT normalise and is silently wrong on a- # non-unit input.+ # quaternion_exp_to_log does not normalise and is invariant to a positive scale (for ||u|| >= eps), for the+ # same reason: atan2(||u||, w) and u / ||u|| do not depend on the scale (q = (1, 2, 3, 4) scaled by 3, 1e-3+ # or 1e3 stays within 2.2e-16 of the unscaled log in float64).quaternion = torch.tensor([1.0, 2.0, 3.0, 4.0], device=device, dtype=dtype)axis_angle = kornia.geometry.conversions.quaternion_to_axis_angle(quaternion)@@ -1760,8 +1761,9 @@ class TestQuaternionExpToLog(BaseTester):# axis-angle vector -- so quaternion_exp_to_log(q) == quaternion_to_axis_angle(q) / 2, but# ONLY on the w >= 0 half. The two functions treat the double cover differently:# quaternion_to_axis_angle collapses q and -q onto the same |theta| <= pi vector, while- # quaternion_exp_to_log takes acos(w) at face value and returns (pi - theta/2) along the- # negated axis for w < 0. The second half of this pin states that divergence explicitly,+ # quaternion_exp_to_log takes the angle atan2(||u||, w) of the quaternion as given (acos(w) for+ # a unit one, u the vector part) and returns (pi - theta/2) along the negated axis for w < 0.+ # The second half of this pin states that divergence explicitly,# because "log is half the axis-angle" is false without the restriction: over 500 random# float64 unit quaternions (seeded torch.Generator(3)) the two agree to 4.44e-16 on the# 243 with w >= 0 and disagree by up to 3.1367975802888637 on the 257 with w < 0.@@ -2376,12 +2378,14 @@ class TestRotationMatrixToAngleAxis(BaseTester):)def test_convention_axis_angle_matrix_roundtrip_is_accurate_in_float64(self, device):- # rotation_matrix_to_axis_angle(axis_angle_to_rotation_matrix(v)) returns v to roundoff,- # except on the low-angle Taylor branch (theta**2 <= 1e-6), whose second-order truncation- # leaves about theta**3 / 6 (1.2e-10 at the 9e-4 row). 1e-9 keeps ~8x headroom over that- # and would catch a first-order branch (theta**2 / 2 = 4e-7). Measured over 2e5 float64- # axis-angles drawn in float64 (uniform, log-spaced small and near-pi angles): 1.6e-10 max,- # at theta just under 1e-3; 1.3e-15 max for theta near pi.+ # rotation_matrix_to_axis_angle(axis_angle_to_rotation_matrix(v)) returns v to roundoff, the+ # low-angle Taylor branch (theta**2 <= 1e-6) included: that branch keeps the theta**2 terms of+ # both series (kornia#4838), so its truncation is theta**5 / 120, below float64 rounding (the+ # second-order branch it replaced left theta**3 / 6, 1.2e-10 at the 9e-4 row, which+ # test_convention_low_angle_branch_matches_quaternion_route_4838 pins). 1e-9 catches a+ # first-order branch (theta**2 / 2 = 4e-7). Measured over 2e5 float64 axis-angles drawn in+ # float64 (uniform, log-spaced small and near-pi angles): 7.9e-18 max on the Taylor branch, at+ # theta just under 1e-3; 1.6e-15 max, for theta near pi.# Snippet used to generate the inputs (stdlib only):# n = math.sqrt(14.0); axis = (1 / n, 2 / n, 3 / n)# [[theta * a for a in axis] for theta in (9e-4, 1e-3, 0.7, 2.0, math.pi)]diff --git a/tests/test_docstring_style.py b/tests/test_docstring_style.pyindex aa9721d64..396739798 100644--- a/tests/test_docstring_style.py+++ b/tests/test_docstring_style.py@@ -20,7 +20,12 @@``testing/docstring_rollout.py`` lists the modules that are converted. For those, this file enforces what a reader,an IDE or a coding agent relies on and what a refactor silently breaks; the shape rule applies everywhere:-* every public function and class has a ``See Also:`` section whose entries name a sibling and say how it differs;+* every public function and class has a docstring, so none escapes the checks below;+* every public function and class has a ``See Also:`` section whose entries name a sibling and say how it differs,+ unless its title line is a status title, ``Alias of :func:`target`.`` or ``Deprecated: use :func:`target`.``,+ which points to another public kornia object and needs none (``Deprecated: use`` only on an object that+ ``kornia.core._compat.deprecated`` wraps, ``Alias of`` only on one that forwards to its target unchanged, as+ ``testing.docstring_style.forwards_to`` checks);* every Python-domain object role (``:func:``, ``:class:``, ``:meth:``, ``:attr:``, ``:data:``, ``:const:``,``:exc:``, ``:obj:``) resolves to a name that Sphinx registers from ``docs/source``, or to a torch, Python or NumPyobject of a type that role links in the cached intersphinx inventories, so it renders as a link and not as plain@@ -57,7 +62,9 @@ from testing.docstring_style import (catalog_unimportable,covered,docstring_roles,+ forwards_to,intersphinx_names,+ is_deprecated_wrapper,is_documented,is_private_target,math_shape_roles,@@ -69,6 +76,8 @@ from testing.docstring_style import (rst_pages,see_also_entries,sphinx_registry,+ status_target,+ undocumented_public_objects,)KORNIA_ROOT = Path(kornia.__file__).resolve().parent@@ -81,6 +90,13 @@ CONVERTED = sorted(((qualified, obj) for obj, qualified in public_objects().values() if covered(obj.__module__, DOCSTRING_MODULES)),key=lambda pair: pair[0],)+UNDOCUMENTED = sorted(+ (+ qualified+ for obj, qualified in undocumented_public_objects().values()+ if covered(obj.__module__, DOCSTRING_MODULES)+ ),+)SHAPE_CHECK_FILES = sorted([*KORNIA_ROOT.rglob("*.py"), *rst_pages(DOCS)])@@ -143,11 +159,41 @@ def test_rollout_lists_are_real() -> None:assert reason.strip(), f"SEE_ALSO_EXCEPTIONS entry {key!r} needs a reason"[email protected]("qualified", UNDOCUMENTED, ids=UNDOCUMENTED)+def test_converted_object_has_docstring(qualified: str) -> None:+ pytest.fail(+ f"{qualified} is exported without a docstring, so the checks cannot see it; write one. A thin wrapper "+ "that only forwards to another object starts with 'Alias of :func:`target`.', a deprecated one with "+ "'Deprecated: use :func:`target`.'"+ )++@pytest.mark.parametrize(("qualified", "obj"), CONVERTED, ids=[qualified for qualified, _ in CONVERTED])def test_converted_object_has_see_also(qualified: str, obj: object) -> None:if qualified in SEE_ALSO_EXCEPTIONS:pytest.skip(f"excepted: {SEE_ALSO_EXCEPTIONS[qualified]}")- entries = see_also_entries(own_docstring(obj))+ doc = own_docstring(obj)+ entries = see_also_entries(doc)+ target = status_target(doc)+ if target:+ # an "Alias of" or "Deprecated: use" title points to another public kornia object, only a deprecated object+ # says "Deprecated: use", and only one that forwards to its target says "Alias of", whether or not a See+ # Also follows+ resolved = resolve_reference(obj, target, REGISTRY)+ message = f"{qualified}: a status title points to another public kornia object, not to {target!r}"+ assert resolved is not None, message+ assert resolved is not obj, message+ assert not is_private_target(target), message+ assert not doc.lstrip().startswith("Deprecated: use") or is_deprecated_wrapper(obj), (+ f"{qualified} has a 'Deprecated: use' title but is not wrapped by kornia.core._compat.deprecated"+ )+ assert not doc.lstrip().startswith("Alias of") or forwards_to(obj, resolved), (+ f"{qualified} has an 'Alias of' title but does not only forward to {target!r}: an alias function returns "+ "the target called with its own arguments, unchanged and with the target's defaults, and an alias class "+ "subclasses it and adds nothing"+ )+ if entries is None:+ return # the title points to the target, so a See Also would only repeat itassert entries, (f"{qualified} needs a 'See Also:' section with at least one bullet like ""'- :func:`name`: how it differs.' (or an entry in SEE_ALSO_EXCEPTIONS with a reason)"diff --git a/tests/test_docstring_style_helpers.py b/tests/test_docstring_style_helpers.pyindex 025837007..d0f8fe556 100644--- a/tests/test_docstring_style_helpers.py+++ b/tests/test_docstring_style_helpers.py@@ -17,6 +17,7 @@"""Tests for the shared docstring-style helpers in ``testing/docstring_style.py``."""+import functoolsimport inspectfrom functools import cachefrom pathlib import Path@@ -28,13 +29,16 @@ from kornia.augmentation.auto.operations.ops import AutoContrastfrom kornia.geometry.boxes import Boxes, Boxes3Dfrom kornia.geometry.depth import DepthWarper, depth_to_3d, depth_to_3d_v2+from testing import docstring_stylefrom testing.docstring_style import (catalog_missing,catalog_required,catalog_unimportable,covered,docstring_roles,+ forwards_to,intersphinx_names,+ is_deprecated_wrapper,is_documented,is_private_target,is_shape_tuple,@@ -45,6 +49,8 @@ from testing.docstring_style import (resolve_reference,see_also_entries,sphinx_registry,+ status_target,+ undocumented_public_objects,)DOCS = Path(__file__).resolve().parents[1] / "docs" / "source"@@ -567,3 +573,244 @@ def test_own_docstring_ignores_whitespace_only():""" """assert own_docstring(blank) is None+++ ("doc", "target"),+ [+ ("Alias of :func:`axis_angle_to_rotation_matrix`.", "axis_angle_to_rotation_matrix"),+ ("Alias of :class:`kornia.geometry.depth.DepthWarper`.", "kornia.geometry.depth.DepthWarper"),+ (+ "Alias of :func:`~kornia.geometry.conversions.rotation_matrix_to_axis_angle`.",+ "kornia.geometry.conversions.rotation_matrix_to_axis_angle",+ ),+ ("Alias of :py:func:`foo`.\n\nMore text.", "foo"),+ (+ "Deprecated: use :func:`~kornia.geometry.conversions.axis_angle_to_quaternion`.",+ "kornia.geometry.conversions.axis_angle_to_quaternion",+ ),+ ("Deprecated: use :class:`kornia.geometry.boxes.Boxes`.", "kornia.geometry.boxes.Boxes"),+ ("Deprecated: use :py:meth:`Foo.bar`.\n\nIt warns on every call.", "Foo.bar"),+ ],+)+def test_status_target_accepts_a_status_title(doc, target):+ assert status_target(doc) == target+++def test_is_deprecated_wrapper_recognises_the_decorator():+ from kornia.core._compat import deprecated++ def plain():+ return 1++ class Plain:+ def __init__(self) -> None:+ pass++ wrapped = deprecated(replace_with="plain", version="0.1")(plain)+ wrapped_class = deprecated(replace_with="Plain", version="0.1")(type("Old", (Plain,), {}))+ assert is_deprecated_wrapper(wrapped)+ assert is_deprecated_wrapper(wrapped_class)+ assert not is_deprecated_wrapper(plain)+ assert not is_deprecated_wrapper(Plain)+++ "doc",+ [+ "Alias of :func:`foo`",+ "Alias of :func:`foo` with a twist.",+ "Alias of :func:`foo` and :func:`bar`.",+ "Alias for :func:`foo`.",+ "Return the thing.\nAlias of :func:`foo`.",+ "Alias of foo.",+ "",+ "Deprecated, use :func:`foo`.",+ "Deprecated: use foo.",+ "Deprecated: use :func:`foo`",+ "Deprecated: use :func:`foo` instead.",+ "Deprecated: use :func:`foo` or :func:`bar`.",+ "Deprecated: Use :func:`foo`.",+ "Deprecated since 0.7.0: use :func:`foo`.",+ "Return the thing.\nDeprecated: use :func:`foo`.",+ "Deprecated: use :math:`foo`.",+ ],+)+def test_status_target_rejects_other_titles(doc):+ assert status_target(doc) is None+++def test_undocumented_public_objects_complements_public_objects():+ documented, undocumented = public_objects(), undocumented_public_objects()+ assert not documented.keys() & undocumented.keys()+ assert all(own_docstring(obj) is None for obj, _ in undocumented.values())+++def test_public_and_undocumented_objects_split_the_export_walk(monkeypatch):+ # kornia may export nothing undocumented, so the split is checked on a synthetic walk. ``__wrapped__`` calls the+ # functions past their own ``functools.cache``, which the patched walk would otherwise never reach.+ def undocumented():+ pass++ def documented():+ """Do the thing."""++ walk = ((undocumented, "m.undocumented"), (documented, "m.documented"))+ monkeypatch.setattr(docstring_style, "_exported_objects", lambda: walk)+ assert [qualified for _, qualified in undocumented_public_objects.__wrapped__().values()] == ["m.undocumented"]+ assert [qualified for _, qualified in public_objects.__wrapped__().values()] == ["m.documented"]+++def _alias_target(a, b=1, *args, c=2, **kwargs):+ return a+++def _forwarding_alias(a, b=1, *args, c=2, **kwargs):+ """Alias of :func:`_alias_target`."""+ return _alias_target(a, b, *args, c=c, **kwargs)+++def _keyword_alias(a, b=1):+ """Alias of :func:`_alias_target`."""+ return _alias_target(a, b=b)+++class _AliasTargetClass:+ pass+++class _AliasClass(_AliasTargetClass):+ """Alias of :class:`_AliasTargetClass`."""+++class _SubclassWithBody(_AliasTargetClass):+ """Alias of :class:`_AliasTargetClass`."""++ def extra(self):+ return 1+++class _TwoBases(_AliasTargetClass, Exception):+ """Alias of :class:`_AliasTargetClass`."""+++def _swapped(a, b):+ return _alias_target(b, a)+++def _constant_argument(a):+ return _alias_target(a, 2)+++def _extra_statement(a):+ a = a + 1+ return _alias_target(a)+++def _dropped_argument(a, b=1):+ return _alias_target(a)+++def _renamed_keyword(a, b=1):+ return _alias_target(a, c=b)+++def _other_target(a):+ return _keyword_alias(a)++[email protected]_cache+def _decorated(a):+ return _alias_target(a)+++def _renamed_positional(x, y=1):+ return _alias_target(x, y)+++def _required_where_target_defaults(a, b):+ return _alias_target(a, b)+++def _changed_default(a, b=5):+ return _alias_target(a, b)+++def _renamed_positional_changed_default(x, y=3):+ return _alias_target(x, y)+++def _defaulted_where_target_requires(a=0):+ return _alias_target(a)+++def _keyword_changed_default(a, c=9):+ return _alias_target(a, c=c)+++def _default_into_target_kwargs(a, d=1):+ return _alias_target(a, d=d)+++ ("obj", "target"),+ [+ (_forwarding_alias, _alias_target),+ (_keyword_alias, _alias_target),+ (_AliasClass, _AliasTargetClass),+ # positional parameters reach the target by position, so their names may differ+ (_renamed_positional, _alias_target),+ # an alias may require what the target defaults: every call it accepts still behaves as the target+ (_required_where_target_defaults, _alias_target),+ ],+)+def test_forwards_to_accepts_a_forwarding_alias(obj, target):+ assert forwards_to(obj, target)+++def test_forwards_to_resolves_a_target_from_the_closure():+ def local_target(x):+ return x++ def local_alias(x):+ """Alias of the local target."""+ return local_target(x)++ assert forwards_to(local_alias, local_target)+++ ("obj", "target"),+ [+ (_swapped, _alias_target),+ (_constant_argument, _alias_target),+ (_extra_statement, _alias_target),+ (_dropped_argument, _alias_target),+ (_renamed_keyword, _alias_target),+ (_other_target, _alias_target),+ (_decorated, _alias_target),+ # a default that differs from the target's changes what a call relying on it returns+ (_changed_default, _alias_target),+ (_renamed_positional_changed_default, _alias_target),+ (_keyword_changed_default, _alias_target),+ (_defaulted_where_target_requires, _alias_target),+ (_default_into_target_kwargs, _alias_target),+ (_SubclassWithBody, _AliasTargetClass),+ (_TwoBases, _AliasTargetClass),+ (_AliasClass, object),+ (len, _alias_target),+ ],+)+def test_forwards_to_rejects_anything_else(obj, target):+ assert not forwards_to(obj, target)+++def test_forwards_to_rejects_a_live_function_titled_alias_of_another():+ # The mutant the status-title check used to accept: give a live converted function the title+ # "Alias of :func:`~kornia.geometry.conversions.rotation_matrix_to_quaternion`." and it skipped its See Also,+ # although it computes something else entirely.+ from kornia.geometry.conversions import axis_angle_to_quaternion, rotation_matrix_to_quaternion++ title = "Alias of :func:`~kornia.geometry.conversions.rotation_matrix_to_quaternion`."+ assert status_target(title) == "kornia.geometry.conversions.rotation_matrix_to_quaternion"+ assert not forwards_to(axis_angle_to_quaternion, rotation_matrix_to_quaternion)