Homography Primer — Describing Planar Correspondence With a Single 3x3 Matrix

Corresponding points on "the same plane" — a tabletop, a poster, a road surface — captured in two images are related by a much simpler relationship than a general 3D scene. No matter where the camera is, points on that plane can be transformed to one another using nothing but a single 3\times3 matrix. This matrix is the homography (a projective transform). Where the Epipolar Geometry Primer deals with correspondence constraints that presuppose scene depth, homography is a contrasting case in that it handles correspondences that don't depend on depth at all — and only by being able to use both appropriately do you get the full picture of two-view geometry.

0. 30-Second Summary

• A homography H is a 3\times3 matrix representing an image correspondence \tilde{\mathbf{x}}'\sim H\tilde{\mathbf{x}}, either for points on the same plane or for a camera undergoing pure rotation. It has 8 degrees of freedom, up to scale ambiguity.
• The DLT (Direct Linear Transform) method builds two linear equations per point correspondence, and solves for H linearly via SVD from 4 or more correspondences. Hartley's normalization is effective for numerical stabilization.
• Because real correspondences include mismatches, outliers are removed with RANSAC before the final estimate. The minimal sample size is 4 points, which keeps the number of robust-estimation iterations lower than for Essential/Fundamental Matrix estimation.
• Given a calibrated camera, H can be decomposed in the form H=K(R+\mathbf{t}\mathbf{n}^\mathsf{T}/d)K^{-1} into rotation R, translation direction, and plane normal \mathbf{n} — though in general, multiple physically plausible candidate solutions remain, and you need additional information to narrow them down.
• For planar scenes or pure rotation, homography is a more appropriate model than the Essential or Fundamental Matrix. Failing to detect this degeneracy means forcibly attempting 3D reconstruction in a situation where recovering depth is fundamentally not possible.

1. What Is Homography: Planar Projective Transform

When homogeneous coordinates \tilde{\mathbf{x}}=(x,y,1)^\mathsf{T}, \tilde{\mathbf{x}}'=(x',y',1)^\mathsf{T} on two images satisfy the relation

via some 3\times3 matrix H, we call H a homography. \sim means equal up to scale — multiplying H by any nonzero constant represents the same transform — so H's degrees of freedom are 9-1=8.

There are broadly two physical conditions under which a homography holds. First, all the corresponding 3D points lie on a single plane. Second, even for a scene with general 3D structure, if the camera translates not at all and only rotates purely (pan/tilt), the relationship can be described by a homography regardless of depth. This is because when the camera only rotates, no parallax arises at all.

2. Estimation via the DLT Method

From a single correspondence (x,y)\to(x',y'), we can derive a linear constraint on each element h_1,\dots,h_9 of H (writing \mathbf{h}=\operatorname{vec}(H)). Expanding the condition that the cross product \tilde{\mathbf{x}}'\times H\tilde{\mathbf{x}}=\mathbf{0} vanishes gives the following two independent equations per correspondence.

With 4 correspondences, you get 8 equations, which (in general position) uniquely determines the 8-degree-of-freedom H. In the realistic case where 5 or more correspondences are available, you find the least-squares solution to A\mathbf{h}=\mathbf{0} for the matrix A stacking all correspondences — that is, the right singular vector corresponding to A's smallest singular value, via SVD. This is the DLT (Direct Linear Transform) method.

Just as with the 8-point algorithm in epipolar geometry, using raw pixel coordinates directly tends to be numerically ill-conditioned. The standard implementation is Hartley's normalized DLT: apply a similarity transform T,T' to each image's point set so it has zero centroid and average distance \sqrt{2}, solve in that normalized frame, then transform coordinates back with H=T'^{-1}H_{\text{norm}}T.

3. Robust Estimation via RANSAC

Since real correspondences include mismatches, applying DLT directly to every correspondence lets outliers badly distort the solution. RANSAC repeats the following:

1. Randomly select 4 correspondences, and build a hypothesis for H via DLT.
2. For every correspondence, compute the reprojection error between the position predicted by H and the actual corresponding point.
3. Adopt the hypothesis with the most correspondences (inliers) within the threshold.
4. Solve DLT once more using all final inliers, and finish with nonlinear optimization (direct minimization of reprojection error) if needed.

The number of iterations needed can be estimated, given an inlier ratio w, minimal sample size s=4, and target success probability p, as

For the same inlier ratio, homography's s=4 requires fewer iterations than Essential/Fundamental Matrix estimation, which needs s=5–8. This is one reason it's common practice, right after SIFT or ORB matching, to first do a coarse geometric verification with homography before moving on to full 3D estimation.

4. Decomposing H: Extracting Rotation, Translation, and Plane Normal

If the camera is calibrated and the intrinsic parameters K_1,K_2 are known, the normalized homography \tilde H = K_2^{-1}HK_1 can be written, using the plane's unit normal \mathbf{n} (in camera 1's coordinate frame), distance to the plane d, and relative pose R,\mathbf{t}, as

If the camera undergoes pure rotation with no translation, \mathbf{t}=\mathbf{0}, so \tilde H=R — the rotation matrix itself.

The process of recovering R,\mathbf{t}/d,\mathbf{n} from \tilde H is called homography decomposition. Several algorithms are known, including the classical Faugeras–Lustman method and the analytic Malis–Vargas method, which obtain a closed-form solution using the eigendecomposition of \tilde H^\mathsf{T}\tilde H. However, purely from the math, as many as 4 physically possible solutions can remain (including ones corresponding to sign flips or reflections). In practice, these are narrowed down using:

• Positive depth (cheirality): the triangulated points must lie in front of both cameras.
• Plausibility of the plane normal: consistency with a rough normal direction already known from the application — such as the ground or a wall.
• Consistency across multiple frames: even if ambiguous in a single frame, tracking over time reveals unnatural solutions as lacking continuity.

OpenCV's decomposeHomographyMat performs this decomposition and provides filter functions (such as filterHomographyDecompByVisibleRefpoints, which selects the solution close to a known plane normal) that help evaluate the multiple candidates.

5. Relationship to Epipolar Geometry: When H Is the Right Answer

As we saw in the Epipolar Geometry Primer, two-view correspondence in a general 3D scene is described by the Fundamental/Essential Matrix. Homography is a special case of that, and the choice between them is as follows.

The problem is that "many inliers for H" and "the scene truly is planar or the camera is truly purely rotating" can sometimes be hard to distinguish from observations alone. Even in a general 3D scene, a wall or table dominating the field of view can strongly fit a homography. ORB-SLAM's initialization process handles this ambiguity by estimating both H and F in parallel via RANSAC, scoring the goodness-of-fit of each, and automatically selecting the model suited to the scene structure and camera motion. The implementation-level key point is using a score that accounts for the difference in each model's degrees of freedom (an idea related to GRIC), rather than simply comparing inlier counts.

6. Applications: Image Stitching, AR Plane Tracking, and Ground-Plane Estimation

Image stitching (panorama composition) — combining multiple images shot by rotating the camera in place into a single image — is a representative application of homography. The homography between adjacent images is estimated, and each is warped into a common reference frame and blended. When the assumption that the camera is nearly purely rotating breaks down (shooting while walking, or a nearby subject in the scene), parallax produces ghosting and doubled images.

Plane-anchor tracking in AR detects a plane, such as a desk or poster, in the first frame, and by tracking the homography to each subsequent frame, can stably determine the relative pose to that plane, frame by frame. Using the decomposed R,\mathbf{t}/d, you can overlay a virtual object anchored to the plane's coordinate frame without visual inconsistency.

Ground-plane estimation exploits the strong prior knowledge that a road or floor surface is "nearly planar." Ground-plane detection in vehicle cameras or robots uses methods that track the homography between consecutive frames and detect regions that deviate from it (obstacles, non-ground objects). This is an approach that detects a breakdown of geometric consistency, rather than recognizing the object itself.

7. Implementation Example in OpenCV

import cv2 as cv
import numpy as np

orb = cv.ORB_create(nfeatures=3000)
kp1, des1 = orb.detectAndCompute(img1, None)
kp2, des2 = orb.detectAndCompute(img2, None)
matches = cv.BFMatcher(cv.NORM_HAMMING).knnMatch(des1, des2, k=2)
good = [m for m, n in matches if m.distance < 0.75 * n.distance]

p1 = np.float32([kp1[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
p2 = np.float32([kp2[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)

# threshold is the allowed reprojection error, in pixels. USAC_MAGSAC is also selectable in place of RANSAC.
H, mask = cv.findHomography(p1, p2, method=cv.RANSAC, ransacReprojThreshold=3.0)
inliers = mask.ravel().astype(bool)

# if K is known, decompose into candidate solutions
num_solutions, Rs, ts, ns = cv.decomposeHomographyMat(H, K)

Note that the H returned by findHomography is scale-ambiguous. As with the Essential Matrix in the Epipolar Geometry Primer, the translation vector obtained from decomposition also determines only a direction — its absolute scale must be supplied by some other means (a known plane distance, a stereo baseline, an inertial sensor, and so on).

8. Difficult Conditions

• Broken planarity: even a scene that looks planar can include objects with real thickness — books, sign edges, plants — and points on them become systematic outliers. Loosening RANSAC's threshold carelessly lets non-planar points get pulled in, distorting H itself.
• The pure-rotation assumption breaking down: if handheld stitching includes even slight translation, closer subjects shift more, producing ghosting. Using a tripod, or rotating near the lens's optical center, is preferable.
• Degenerate configurations: if the correspondence points concentrate along a single line in the image or within a narrow region, the DLT matrix becomes ill-conditioned, and error spikes sharply in the extrapolated regions of H — areas away from the correspondence points.
• Repetitive patterns or low-texture planes: with a repetitive pattern like tiled flooring or a lattice window, local descriptors alone can't distinguish a correct correspondence from a mismatch shifted by one period.
• Ambiguity in decomposition: if K is inaccurate, or noise is large, it can be impossible to uniquely pick out the physically correct solution among the multiple candidates from decomposition. Always combine this with additional prior knowledge (normal direction, positive depth).

9. Summary

Homography is a framework that exactly represents two limited but practically frequent situations — correspondence on a plane, or a camera undergoing pure rotation — with a single 3\times3 matrix. The DLT method is the least-squares starting point, RANSAC is the countermeasure for outliers, and decomposition is the final stage that extracts the physical rotation, translation, and normal. Above all, what matters is judging when homography is the right model, and when you should switch to the Fundamental/Essential Matrix — get that boundary wrong, and you end up trying to recover depth that doesn't exist, in a scene that's nothing but a plane.

References

• Hartley & Zisserman, Multiple View Geometry in Computer Vision (authors' official page)
• Fischler & Bolles, Random Sample Consensus (Communications of the ACM, 1981)
• Malis & Vargas, Deeper Understanding of the Homography Decomposition for Vision-based Control (INRIA Research Report RR-6303, 2007)
• Mur-Artal, Montiel & Tardós, ORB-SLAM: A Versatile and Accurate Monocular SLAM System (IEEE TRO, 2015)
• OpenCV — Basic concepts of the homography explained with code
• OpenCV — findHomography / decomposeHomographyMat reference
• OpenCV — Image Stitching module