How images reveal features, motion, geometry, and camera pose, connected to Visual SLAM and its surrounding methods.
This is an editorially selected reading order. Begin at step 1 or go directly to the topic you need.
Camera calibration is the process of recovering the intrinsic parameters and lens distortion needed to map pixel coordinates onto 3D space. From revisiting the pinhole model, through radial and tangential distortion models, Zhang's planar-pattern method, reprojection-error optimization, to what breaks downstream in Visual SLAM and VO when calibration drifts — a systematic walkthrough with equations and diagrams.
Feature detection selects "places you can find again" in an image and uses them for matching. This article connects the principles of corners, blobs, DoG, FAST, ORB, and SIFT with robustness, evaluation, current tools, and learning-based methods into a single coherent picture.
Feature tracking is the process of matching the same physical point across frames in a video sequence. This article organizes template tracking, Lucas–Kanade optical flow, descriptor matching, RANSAC, outlier rejection, and Kalman prediction with equations and implementation steps, and explains the conditions under which Visual Odometry and SLAM break down, along with mitigations.
Optical Flow is a vector field describing which direction each pixel in a sequence of images has moved. This article works through the brightness-constancy constraint, Lucas–Kanade and Horn–Schunck, pyramids, dense flow versus sparse feature tracking, learning-based methods such as RAFT, and camera motion, dynamic objects, and evaluation methodology, with the underlying math.
A homography is the relation that ties together points on the same plane, or the view of a purely rotating camera, using a single 3x3 projective transform. This article covers DLT-based estimation, robustification with RANSAC, decomposition into rotation, translation, and plane normal, and when to use it versus epipolar geometry, alongside applications like image stitching and AR plane tracking, with equations and diagrams.
Epipolar geometry is the theory that constrains how the same 3D point projects into two images. This article systematically explains the Fundamental Matrix and Essential Matrix, the 8-point and 5-point algorithms, RANSAC, triangulation, the choice between epipolar geometry and homography, calibration, OpenCV/COLMAP implementation, failure conditions, and recent learning-based estimation, with equations and diagrams.
Perspective-n-Point (PnP) is the geometric problem of estimating a camera's position and orientation from known 3D points and their corresponding image points. This article works through the projection equation, P3P/AP3P, EPnP, iterative optimization, RANSAC, planar degeneracy, and how PnP is used in Visual SLAM, AR, and surveying, with the math and an implementation checklist.
Structure from Motion (SfM) recovers both a scene's 3D structure and every camera pose simultaneously from a set of photos whose order and shooting positions are unknown. This article systematically covers feature matching, geometric verification, the two reconstruction strategies of Incremental and Global SfM, triangulation, the relationship to Bundle Adjustment, and the difference from Visual-SLAM, alongside current implementations like COLMAP.
Bundle Adjustment is a nonlinear least-squares problem that moves many camera poses and 3D points at once to minimize the reprojection error across all observations. This article systematically covers why the Jacobian is sparse, why the Schur complement makes large-scale problems solvable, the role of the Levenberg-Marquardt method, gauge freedom, and the relationship to Pose Graph optimization and SfM, with equations and diagrams.
Multi-View Stereo (MVS) recovers dense 3D shape from many images using known camera poses. This article systematically covers the core principle of photo-consistency, the two classical approaches of Plane-Sweep and Patch-based (PMVS), deep-learning-based methods epitomized by MVSNet, and meshing via depth-map fusion, alongside the relationship to stereo and depth cameras, with equations and diagrams.
Visual Odometry (VO) and Visual-Inertial Odometry (VIO) estimate the relative motion of a robot or camera from a sequence of images and inertial measurements. This article works through monocular scale ambiguity, IMU preintegration, initialization, filtering versus optimization, implementations such as VINS-Mono and OpenVINS, evaluation metrics, and failure conditions, with math and diagrams.
Loop closure is the process of recognizing that a robot has returned to a previously visited place, and distributing the accumulated self-localization error across the whole map. This article systematically explains place recognition, image retrieval, geometric verification, pose-graph optimization, defenses against false positives, and current SLAM implementations and research.
With no GPS and no map, a robot can still find its own position and build a map of its surroundings from camera images alone. From feature tracking through epipolar geometry, the difference between Visual Odometry and SLAM, the lineage of landmark algorithms from ORB-SLAM to LSD-SLAM and DSO, and Bundle Adjustment, a systematic ground-up tour of how Visual-SLAM actually works.