How Depth Cameras Work and Key Products — RealSense, Kinect, Orbbec

A depth camera captures the distance (depth) to an object on a per-pixel basis. Where a stereo camera is a passive design that detects disparity using visible light alone, most depth cameras actively project some form of light onto the object and derive depth from what comes back. Implementations fall broadly into three categories: "active stereo," which projects a random dot pattern with an infrared emitter and does stereo matching on it; "structured light," which computes depth geometrically from how a known pattern distorts; and "Time of Flight" (ToF), which computes distance directly from the round-trip time (or phase shift) of light. All three share the same underlying strength — because they can use an artificial cue, they can capture depth even on targets like a plain white wall or a solid-colored floor, where passive stereo struggles to detect disparity. They're used for indoor robot SLAM, VR/AR hand tracking, smartphone facial authentication, and industrial 3D measurement.

Kinect (2010, on display at E3)

Intel RealSense D435

Images: Kinect and Xbox 360 at E3 2010 (James Pfaff, CC BY 2.0) / Intel Realsense depth camera D435 (Marc Auledas, CC BY-SA 4.0), both Wikimedia Commons. The RealSense photo shows the base D435, not the IMU-equipped D435i discussed in the text.

Principle: three active approaches

Active stereo has an infrared projector cast a random dot pattern onto an object, then does stereo matching between two infrared cameras on the shift in that pattern to compute depth. The underlying principle is the same triangulation as passive stereo (Z = fB/d, see the stereo camera article), except the texture is artificially added. Intel's RealSense D400 series uses this approach.

Structured light triangulates using a pair consisting of "a projector emitting a known pattern" and "a single camera," instead of two cameras. There's a baseline between the projector and the camera too, and depth is back-calculated by comparing how the projected pattern (dots, stripes, etc.) distorts against object surfaces to a known reference pattern. The original Kinect (2010) and Apple's TrueDepth (iPhone Face ID) both use this approach — TrueDepth projects more than 30,000 infrared dots onto a face to read its 3D shape.

ToF (Time of Flight) applies the same round-trip-time-of-light principle used by LiDAR, in parallel, per pixel. Beyond directly measuring a pulse's round-trip time (dToF), the approach that dominates consumer and industrial depth cameras is iToF: it projects modulated continuous light and derives distance from the phase shift \Delta\phi between the emitted and reflected signal. With modulation frequency f_{mod} and the speed of light c, distance Z is given by

Because it needs no triangulation, it doesn't need a baseline between a projector and a camera, making it easier to miniaturize and eliminating moving parts. Microsoft's products from Kinect v2 onward, and Orbbec's Femto Bolt, use this approach. All three approaches rely on infrared light, so they share a common weakness: accuracy tends to degrade outdoors under strong sunlight, which is rich in infrared.

Key product specification comparison

The D435i is the base D435 with a 6-axis IMU (Bosch BMI055) added. Its IMU data is output time-synchronized (timestamp-aligned) with the depth data, which is exactly what lets it correct for the camera's own motion while processing depth and point clouds. The original Kinect was built on structured-light technology developed by the Israeli startup PrimeSense; its successor, Kinect v2, switched to ToF. Azure Kinect went out of production in 2023, but Microsoft licensed its time-of-flight technology to Orbbec, and the successor Femto Bolt uses the same depth camera module as Azure Kinect, easing migration for existing applications. The iPhone's TrueDepth is a separate commercial branch of that same PrimeSense structured-light lineage — a real example of one technological root splitting into two utterly different applications: "game console motion sensing" and "smartphone facial authentication."

How One Technology, PrimeSense, Branched Into the Xbox, the iPhone, and Industrial Robots

The Israeli company PrimeSense that created Kinect — The core technology in the original Kinect (released 2010) wasn't developed in-house at Microsoft — it was structured-light depth-sensing technology built by the Israeli startup PrimeSense. Kinect quickly outgrew its gaming-console origins and found wide use in academic and commercial robotics: the combination of sub-720p resolution with simultaneous depth and RGB capture, at a fraction of the cost of expensive LiDAR, was a remarkable deal for robotics research at the time. A 2011 paper by Richard Newcombe, Shahram Izadi, and colleagues at Microsoft Research, "KinectFusion: Real-time dense surface mapping and tracking" (IEEE ISMAR 2011), demonstrated real-time dense 3D surface mapping using nothing but a single low-cost depth camera and a commodity GPU, becoming one of the founding works of RGB-D SLAM research.

Apple's PrimeSense acquisition and its rebirth as Face ID — In November 2013, Apple acquired PrimeSense for $360 million. That didn't erase the original Kinect's structured-light technology from the market — its technical lineage went on to become the core of Face ID, materializing in the iPhone X's "TrueDepth" camera system (an infrared flood illuminator, a dot projector, and an infrared camera) announced in 2017. It's an unusually large-scale case of technology repurposing: a technology built for game-console motion sensing became, seven years later, the core biometric authentication technology shipping on smartphones at a scale of a billion units.

Microsoft discontinuing Kinect "again" — Over more than a decade, Microsoft has repeatedly launched and then discontinued generation after generation of Kinect-family products: Kinect for Xbox 360, Kinect for Xbox One, Kinect for Windows v2, and the industrial/research-oriented Azure Kinect DK. The Azure Kinect DK's discontinuation, announced in August 2023, was the latest (and so far last) "end of Kinect" in this lineage. But this time it wasn't a full retreat — Microsoft chose to license its ToF technology to Orbbec instead, and Orbbec's "Femto Bolt" arrived carrying forward the same depth camera module as Azure Kinect. It's a pattern echoing the PrimeSense acquisition a decade earlier: one technology surviving across a corporate handoff.

Parameters That Determine Performance

• Depth range: D435i's roughly 0.3–3m range (up to about 10m-equivalent under some conditions) is a near-to-mid-range design, suited to detecting obstacles on a tabletop or around shelving for an indoor robot, or close-range use like hand tracking. Outdoor long-range monitoring is better served by a stereo camera or LiDAR
• Choice of method (active stereo / structured light / ToF): Active stereo carries the computational cost of stereo matching but can be built from relatively inexpensive components. Structured light offers high accuracy at close range, suiting very-near, high-precision applications like Face ID, but tends to face miniaturization constraints from needing a baseline between the projector and camera. ToF needs no moving parts or baseline, making it easier to miniaturize, though it tends not to match structured light's close-range accuracy. Femto Bolt fitting 1024x1024 depth resolution into a compact body is a design that leverages exactly this miniaturization advantage of ToF
• Built-in IMU and time synchronization: For applications where the camera itself moves (handheld, drone-mounted, mobile robot), how precisely the depth data and IMU data are time-synchronized determines the accuracy of motion-blur correction. The built-in IMUs on the D435i, Azure Kinect, and Femto Bolt are designed specifically with this use case in mind
• Tolerance to ambient light: All three infrared-based approaches are stable indoors, but accuracy tends to degrade outdoors under strong sunlight, which washes out the projected pattern or reflected signal. Applications assumed to run outdoors typically need to fall back on passive stereo (ZED, etc.) or a mix of approaches
• Object texture: For indoor robotics and VR/AR dealing with texture-poor targets — a white wall, a solid-colored floor — an active method that can artificially project an infrared pattern has the advantage. That advantage shrinks in texture-rich outdoor environments
• Minimum ranging distance: For applications needing depth at very close range — hand tracking, facial authentication — the lower bound of the depth range directly determines usability. TrueDepth being purpose-built for a target just tens of centimeters away (a face) is the archetypal example of this

References

• Intel RealSense D435i product page
• Kinect (Wikipedia)
• Azure Kinect (Wikipedia)
• Microsoft announces Azure Kinect DK end of production (Microsoft Tech Community)
• Orbbec Femto Bolt product page
• Orbbec x Microsoft collaboration overview
• Microsoft can't stop discontinuing Kinect (TechCrunch)
• Coverage of Apple's PrimeSense acquisition
• How iPhone TrueDepth/Face ID works (Structure)
• KinectFusion paper (IEEE ISMAR 2011)
• Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect (arXiv:1505.05459)