An SBC (Single Board Computer) is a compact computer that integrates a CPU, memory, storage interfaces, and various I/O onto a single board — in an autonomous mobile robot or drone, it's the "brain" behind self-localization, path planning, and sensor processing. Far from being a mere small PC, the design philosophy around how to pack in GPU or NPU (Neural Processing Unit) capability for inference now varies sharply by product line, and three currents run in parallel: Raspberry Pi's general-purpose Linux SoC, NVIDIA Jetson's GPU-integrated AI computer, and Google Coral's dedicated ASIC accelerator.

Raspberry Pi 5 boardRaspberry Pi 5
NVIDIA Jetson Orin NX module on displayNVIDIA Jetson Orin NX (Computex 2025)

Images: Raspberry Pi 5 Board (SimonWaldherr, CC BY-SA 4.0) / Nvidia Jetson Orin NX on a motherboard Computex 2025 (4300streetcar, CC BY 4.0), both Wikimedia Commons. Not the exact Jetson Orin Nano Super compared in the text, but an Orin NX module from the same Orin generation and Ampere architecture. No clean photo of Google Coral (Dev Board/USB Accelerator) turned up on Wikimedia Commons, so no product photo is included for it in this article.

Principle: the TOPS metric and the "memory wall"

When discussing SBC performance, CPU clock speed alone no longer tells the story. The performance of a GPU/NPU specialized for neural network inference is expressed as the number of quantized integer operations (usually INT8) executed per second, in units of TOPS (Tera Operations Per Second — one trillion operations per second). TOPS can be roughly estimated from the number of parallel multiply-accumulate (MAC) units N_{MAC} and the clock frequency f:

\text{TOPS} = N_{MAC} \times f \times 2 \times 10^{-12}

A multiply-accumulate counts as two operations — one multiply plus one add — hence the factor of 2. NVIDIA's Jetson Orin Nano Super, for example, drives 1024 CUDA cores and 32 Tensor cores at around 1.7GHz, reaching 67 TOPS in its power-ceiling-raised "MAXN Super" mode (25W).

Most of this TOPS figure assumes 8-bit integer (INT8) arithmetic rather than 32-bit floating point (FP32). Quantizing the FP32 weights used during training down to INT8 before inference theoretically cuts model size and the required memory bandwidth to a quarter, letting the same compute units push through more parallel operations. It's a design that trades inference precision for maximum throughput within a limited power and memory-bandwidth budget — and Jetson, Hailo, and Coral all quote catalog TOPS figures that assume this INT8 quantization.

That said, the peak catalog TOPS doesn't automatically translate into effective performance. However fast the compute units are, they sit idle if the data they need (weights, intermediate features) can't be fetched from memory fast enough. The roofline model captures this: effective performance P is capped at the smaller of the peak compute throughput P_{peak} and the memory bandwidth BW multiplied by arithmetic intensity (operations per byte) AI.

P = \min(P_{peak},\ AI \times BW)

The Raspberry Pi 5's LPDDR4X-4267, on a 32-bit bus, tops out at a theoretical bandwidth of about 17.1GB/s, versus the Jetson Orin Nano Super's LPDDR5 at 102GB/s — roughly 6x. This rarely shows up as a felt difference in CPU-bound workloads, but for neural network inference that re-reads large weight matrices every frame, this bandwidth gap becomes the rate-limiting factor determining how much of the catalog TOPS figure you can actually extract.

Three design philosophies: general-purpose SoC, GPU-integrated, dedicated ASIC

Products called "SBCs" broadly split into three camps based on how they handle AI inference.

General-purpose SoC (Raspberry Pi) integrates a CPU, GPU, and various I/O controllers, but doesn't ship a dedicated NPU for neural network inference. It runs a full Linux distribution, and its biggest strength is being able to use nearly all of an existing software stack — including ROS 2 — as-is; the design assumes that whenever inference performance is needed, an external accelerator (Hailo, discussed below, for example) fills the gap.

GPU-integrated AI computer (NVIDIA Jetson) integrates a CUDA-programmable GPU and Tensor cores into the same package as high-bandwidth memory. Unlike a dedicated ASIC that can only push pre-quantized, fixed computation graphs, it retains the versatility to run arbitrary CUDA kernels and PyTorch/TensorRT models directly, while offering vastly more parallel compute. Product lines exist for automotive-grade reliability too, such as the AGX Orin, which holds ISO 26262 ASIL-D functional safety certification for autonomous driving and robotics.

Dedicated ASIC accelerator (Google Coral / Hailo) takes the form of a fixed-function ASIC specialized purely for quantized neural network inference, added to a host SBC via USB or M.2. In exchange for sacrificing versatility, it can achieve efficiency (TOPS per watt) that far exceeds the other two approaches.

These three categories aren't mutually exclusive, and real robot projects often combine them. Pairing a general-purpose SoC like the Raspberry Pi 5 with an ASIC accelerator (AI HAT+) is a compromise aimed at getting both the breadth of the Linux/ROS 2 software stack and the power efficiency of offloading inference to an ASIC — and, as the benchmarking paper discussed below shows, this combination can approach the efficiency of a standalone GPU-integrated computer.

Key product specification comparison

Product Type Compute cores AI performance Memory Power Price
Raspberry Pi 5 (16GB) General-purpose SoC (no NPU) Cortex-A76 4-core, 2.4GHz — (external needed) LPDDR4X 16GB (~17.1GB/s) 5V/5A supply (25W max) $305 (as of April 2026)
Raspberry Pi AI HAT+ (Hailo-8) Dedicated ASIC (RPi5 add-on HAT) Hailo-8 NPU 26 TOPS (INT8) — ~2.5W $110
Google Coral USB Accelerator Dedicated ASIC (USB) Edge TPU 4 TOPS (INT8) — 2W ~$75–99
NVIDIA Jetson Orin Nano Super GPU-integrated AI computer Cortex-A78AE 6-core, 1.7GHz + Ampere GPU, 1024 cores/32 Tensor cores 67 TOPS (INT8, MAXN Super mode) LPDDR5 8GB (102GB/s) 7–25W (variable) $249
NVIDIA Jetson AGX Orin (64GB) GPU-integrated AI computer (flagship) Cortex-A78AE 12-core + Ampere GPU, 2048 cores/64 Tensor cores 275 TOPS (INT8) LPDDR5 64GB, 256-bit 15–60W (variable) —

The Raspberry Pi 5 is the model the newbot project uses as its main control computer, handling edge-side processing such as self-localization, safety supervision, and sensor acquisition. Because newbot's LiDAR-Inertial Odometry (FAST-LIO) is an algorithm that runs entirely on the CPU, and the system doesn't need to constantly run large-scale neural network inference, a general-purpose SoC with no dedicated NPU is enough to meet its requirements. Conversely, a robot built around real-time object detection or segmentation from camera feeds would tend to require an integrated GPU like Jetson, or a Hailo/Coral-class external accelerator. The Jetson Orin Nano Super also has an unusual history: a December 2024 software update called "Super Mode" raised the power ceiling and took it from the previous generation's 40 TOPS at 499 to 67 TOPS at249 — more performance at half the price.

History: nostalgia for the BBC Micro, and a gaming GPU maker's pivot

Raspberry Pi: triggered by a drop in Cambridge admissions — Raspberry Pi Foundation founder Eben Upton got his hands on a secondhand BBC Micro (a home computer the BBC deployed for education in the 1980s) at age 10 in 1988, and it was his first taste of programming in BASIC. Later, while serving as Director of Studies for computer science at Cambridge University around 2006, he noticed a surge of applicants arriving with little programming experience, alongside a decline in applications to the Computer Science Tripos itself. Seeing the disappearance from homes of "power-it-on-and-start-programming" computers like the BBC Micro as one cause, Upton spent evenings and weekends from 2006 to 2011 developing a low-cost SBC alongside his day job at Broadcom. The Raspberry Pi Foundation was formally established in 2009, with the goal of a "programmable computer for under $35." In 2011, people from the Foundation brought a prototype to the BBC hoping to carry the "BBC Micro" name forward, which legal circumstances ruled out — but a BBC journalist instead published a demo video of the prototype that went viral, proving there was strong demand well before the official launch.

Jetson: a gaming GPU maker enters robotics in 2014 — In March 2014, NVIDIA CEO Jensen Huang personally unveiled the Jetson TK1 in a keynote at GTC (GPU Technology Conference). It paired the mobile-oriented Tegra K1 chip with a 192-core Kepler-architecture GPU, delivering 326 GFLOPS — an outlandish figure for an embedded device at the time — and brought the CUDA programming model along with it. From the outset, use cases like robot navigation and drone obstacle avoidance were part of the pitch, marking NVIDIA's real entry into the embedded robotics market from a business that, until then, had fought mainly in gaming and datacenter GPUs.

The Jetson lineup went on to refresh its GPU architecture roughly every other generation: the 2017 TX2, with a 256-core Pascal GPU delivering 2x the TK1's performance; the 2018–2020 Volta-architecture Xavier generation, claimed to deliver over 20x the performance and more than 10x the power efficiency of the TX2; and the 2022 Ampere-architecture Orin generation. The consistent pattern is architectures debuted first in the gaming GPU market getting repurposed for embedded use a few years later — a structure in which the technical accumulation of NVIDIA's datacenter and PC GPU business feeds directly into the performance gains of its embedded robotics products, and this is Jetson's core strength.

A real deployment: one home NVR driving multiple Coral Edge TPUs

A representative case of a dedicated ASIC accelerator being run "several at once" is Frigate NVR, an open-source recording and object-detection package. Frigate analyzes surveillance camera feeds in real time and offloads person/vehicle detection to Coral Edge TPUs, with an architecture that assigns one dedicated process per TPU and distributes detection requests from multiple cameras through a shared queue to those processes.

A single Coral USB Accelerator (4 TOPS) is reckoned to handle real-time detection for about two to four 1080p streams on its own; a configuration pairing two 8-TOPS Coral Dual Edge TPU (M.2) cards has reportedly handled 20+ cameras simultaneously while chip utilization stayed around 12%. On the other hand, it's also known that stacking multiple USB Accelerators can run into the host's USB bus bandwidth as a bottleneck, so performance doesn't scale linearly with device count. Adding several dedicated ASICs to a single NVR server to scale up camera count is a textbook deployment pattern that leverages the low cost and power draw per TOPS.

How AI-datacenter demand drove up hobbyist SBC prices

In 2026, Raspberry Pi found itself raising prices repeatedly for reasons entirely outside its own control. The cause was a memory-market crunch driven by surging demand from AI datacenters — Raspberry Pi Foundation CEO Eben Upton himself disclosed on the company blog that the procurement cost of the LPDDR4 DRAM used in the Raspberry Pi 4/5 had jumped sevenfold in a single year. Following a December 2025 price hike, several more increases were announced in the first half of 2026, and an April 2026 hike added 100 to the 16GB model, taking it to305 — more than 2.5x its price at launch (October 2024, when the 16GB model debuted at 120). Ironically, in step with this price surge, the Foundation also introduced a new 1GB model in 2026 priced at just45 — a case where demand for high-performance AI memory reached all the way into the price structure of a completely unrelated market: hobbyist embedded computers.

Coral, abandoned by Google, kept alive by the community

The Edge TPU inside Coral follows the same "systolic array" compute architecture Google developed for its datacenter TPUs — it's effectively a miniaturized version. Where the datacenter-scale TPU v1 handles training and inference with a massive 256×256 systolic array, the Edge TPU shrinks that array down dramatically and pares power consumption to single-digit watts, enabling it to run inside embedded devices with tight power budgets. Google launched Coral in 2019 with this Edge TPU (4 TOPS, 2W) inside a Dev Board and a USB Accelerator, and it built a following as a bolt-on accelerator for existing SBCs like Raspberry Pi, on the strength of power-efficient inference that could run MobileNet v2 at roughly 400 frames per second. Since then, though, Google has effectively stopped investing in the product line without shipping any notable new hardware, and most of the supporting libraries haven't been updated since around 2022. The GASKET driver required by PCIe-connected Coral devices remains unsupported on current Linux kernels. What's kept Coral from falling completely out of use is that Google open-sourced its drivers and firmware; projects like the aforementioned Frigate NVR have picked up the support burden themselves and kept it running in production — a case where official end-of-support and continued real-world use coexist, illustrating the mismatch between hardware product cycles and open-source software lifespans.

Does bolting an accelerator onto an SBC beat Jetson? A 2026 benchmarking paper

An April 2026 paper on running large language model (LLM) inference on edge SBCs, "Cloud to Edge: Benchmarking LLM Inference on Hardware-Accelerated Single-Board Computers" (Renney, Trad, Mattarock, Evetts, Wood), compared power draw and throughput across four configurations: a bare Raspberry Pi 5 (CPU only), a Raspberry Pi 5 with an AI HAT+ (Hailo-10H NPU), a Jetson Orin Nano Super (CPU and GPU configurations separately), and an M5Stack LLM Module (AX630C NPU). The results: adding a Hailo NPU to the Raspberry Pi 5 improved tokens-generated-per-watt efficiency by 9.57x to 39.97x over the CPU-only configuration, dropping energy consumption per megatoken from 27–77 megajoules down to 0.88–5.51 megajoules. In raw throughput, the Jetson Orin Nano's GPU configuration was fastest at 9–10 tokens per second, but the Raspberry Pi 5 + Hailo configuration matched the throughput of Jetson's CPU-only configuration at roughly half the power draw and in a smaller footprint — data backing up the claim that a "general-purpose SoC plus dedicated ASIC" combination can go toe-to-toe with a standalone GPU-integrated computer on power efficiency.

Parameters that determine performance

References

#Raspberry Pi #Jetson Orin #Coral #SBC #Embedded #Robotics