Visual-SLAM is the family of techniques that performs self-localization and mapping relying on camera footage alone (or a camera-plus-IMU combination). It doesn't need an expensive ranging sensor like LiDAR, but it carries a weakness — sensitivity to lighting changes and texture-poor scenes — and the history of overcoming that weakness is essentially the technology trend itself.

OpenCVOpenCV

Image: OpenCV logo, Wikimedia Commons (Apache 2.0)

The Feature-Point-Based Endpoint: ORB-SLAM3

One endpoint of classical Visual-SLAM is ORB-SLAM3. It can run Visual, Visual-Inertial, or multi-map SLAM in real time across monocular, stereo, or RGB-D setups alike, finding correspondence between images using a feature descriptor called ORB, then optimizing pose and map points via bundle adjustment. What bundle adjustment minimizes is the sum of the mismatch (reprojection error) between where a 3D map point \mathbf{X}_i lands when projected into the image under camera pose (\mathbf{R}_j, \mathbf{t}_j), and where the corresponding feature point \mathbf{u}_{ij} was actually observed in the image.

\min_{\{\mathbf{R}_j, \mathbf{t}_j\}, \{\mathbf{X}_i\}} \sum_{i,j} \left\| \pi\left( \mathbf{R}_j \mathbf{X}_i + \mathbf{t}_j \right) - \mathbf{u}_{ij} \right\|^2

Here \pi(\cdot) is the projection function from a 3D point to the image plane, based on the camera's intrinsic parameters. Optimizing this expression jointly over both camera pose and map points is the essence of bundle adjustment, solved within the same nonlinear least-squares framework as the graph optimization covered earlier (see "Technology Trends in SLAM" for details). Years after its release, it's still cited in research papers as a comparison baseline — the de facto standard reference implementation.

End-to-End via Deep Learning: DROID-SLAM

There's also a well-established trend of replacing the explicit process of feature extraction and matching with deep learning outright. DROID-SLAM is the leading example, estimating camera pose and per-pixel depth through recurrent iterative updates and a dense bundle-adjustment layer. Despite being trained on monocular video only, it's flexible enough to also exploit stereo or RGB-D video at test time for better accuracy. That said, its compute demands are steeper than classical methods — inference alone requires 11 GB or more of GPU memory.

Inside DROID-SLAM: Correlation Volumes and Differentiable Bundle Adjustment

Looking a bit more closely at how DROID-SLAM estimates pose and depth without going through explicit feature extraction: it first computes dense feature maps at 1/8 the input resolution, using two CNNs — one for feature extraction, one for contextual information. For each pair of frames, it builds a "4D correlation volume" by exhaustively computing the dot product between every pair of feature vectors, which becomes the foundation for estimating pixel-wise correspondence.

The estimation itself is handled by an iterative update operator. Features from the correlation volume, together with the residual (correction) from the previous iteration, pass through convolutional layers and into a ConvGRU (a convolutional gated recurrent unit), which outputs a correction to the flow (apparent motion). The gating mechanism — which lets the network selectively control what information to incorporate and what to discard — is the part this network has actually learned.

The output of this iterative update feeds into a differentiable Dense Bundle Adjustment (DBA) layer that updates camera pose and per-pixel inverse depth simultaneously. Baking geometric constraints explicitly into the network this way gives even a monocular-trained model the flexibility to handle stereo or RGB-D input without retraining.

Reinventing Map Representation: 3D Gaussian Splatting and NeRF

The fastest-moving area right now is a reinvention of the map's representation format itself. Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) can represent a photorealistic 3D scene as a compact set of parameters, rather than a plain point cloud. That property is steadily lifting SLAM's mapping accuracy, rendering quality, and computational efficiency all at once.

Well into 2026, a stream of new 3DGS/NeRF-based methods have kept appearing at major conferences and journals: Splat-SLAM, which does global optimization from RGB video alone; Gaussian-LIC/Gaussian-LIC2, which fuse LiDAR, IMU, and camera; GTS-SLAM, aimed at harsh environments like underground mines; MTE-SLAM (ICRA 2026), oriented toward semantic SLAM; and PMET-SLAM, which balances photorealistic mapping with efficient tracking. Compared against traditional methods, rendering quality and map reusability (the ability to re-render the environment from an arbitrary viewpoint after the fact) are cited as clear strengths.

The 2025-2026 Current: SLAM Built on Feed-Forward 3D Foundation Models

Taking DROID-SLAM's design — iterative estimation plus differentiable bundle adjustment — a step further, MASt3R-SLAM (a CVPR 2025 Highlight paper), which appeared in late 2024, treats the output of MASt3R — a foundation model pretrained on two-view 3D reconstruction and correspondence — as "prior knowledge." It handles pointmap matching, camera tracking, local map fusion, and loop-closure detection within a single framework even when camera intrinsics are unknown, running at 15 FPS with known camera models while reportedly achieving state-of-the-art accuracy across multiple benchmarks.

This foundation-model line gained further momentum with the 2025 arrival of VGGT (Visual Geometry Grounded Transformer) — a feed-forward 3D foundation model that outputs camera pose, depth, and point clouds simultaneously from multi-view images in a single forward pass. A wave of derivative research built on VGGT's output has followed through 2026. VGGT-SLAM++ (April 2026) turns VGGT's transformer output into a complete SLAM system by combining it with frequent local optimization — via covisibility graphs and visual place recognition — to stabilize trajectories. VGGT-Align (August 2026) resolves chunk-to-chunk scale drift over long sequences by constraining scale using dominant geometric invariants extracted from each chunk's point cloud. Analysis work has also emerged, such as Co-VGGT (July 2026), which showed that VGGT implicitly encodes covisibility (which viewpoints share the same region) without any explicit supervisory signal — an active effort to understand what these foundation models' internal representations actually capture.

What the Benchmarks Show: More Accurate, But Not a Silver Bullet

A cluster of studies quantifying foundation-model-based SLAM's real-world performance appeared in the latter half of 2026. An evaluation comparing five monocular SLAM systems on high-altitude nadir footage from DJI drones found that MASt3R-SLAM achieved the lowest mean horizontal absolute error at 0.53% of path length, while noting that vertical (altitude) accuracy remains an open problem. A separate study (August 2026) evaluating monocular SLAM under both synthetic and real-world corruptions found that while classical feature-based methods tend to suffer "catastrophic tracking failure" under degraded conditions, learned trackers largely just replace that catastrophic failure with "sustained, and sometimes severe, drift." In other words, learned methods may trade a dramatic, visible failure mode for a quieter one where error accumulates unnoticed — a reminder that accuracy metrics alone don't tell the whole story of which approach is actually better.

On scale ambiguity — the classic weakness where a monocular feed alone can't recover absolute distance scale — new approaches are emerging too. Scalix (August 2026), which integrates learned depth prediction into a probabilistic factor graph, reports state-of-the-art performance on both metric (absolute-scale) and up-to-scale (relative-scale) benchmarks.

Practical Selection Criteria

The practical breakdown at this point looks like this:

None of these families is really replacing the others — the current reality is that they're settling into a division of use cases.

References

#Visual-SLAM #ORB-SLAM3 #Gaussian Splatting