Edge computing hardware is a stack of tiers, not a single box. At the bottom sit sensors and microcontroller-class devices. Above them are gateways, embedded PCs and hardened small servers, and nearest the network are server clusters or micro data centres. Accelerators (GPUs, FPGAs, TPUs) are added to gateways and servers as the workload needs. Raw compute is not all that makes this hardware edge-ready. The hardware has to survive the site: tight spaces, vibration or humidity, limited power and connectivity, and no IT staff nearby to recover it. Edge is also not automatically faster than cloud. It wins only while its lower network delay outweighs the extra queuing delay that comes from small, unpooled capacity. Picking and sizing an accelerator depends on the model family, the precision the device can execute, the input and batch size, and the enclosure’s thermal envelope.

Edge computing hardware, layer by layer

The Linux Foundation’s LF Edge taxonomy describes edge computing as putting compute and storage “as close as both necessary and feasible” to where data is generated and acted on. Where each resource sits is a trade between the benefits of centralisation and decentralisation. Its general rule is that the closer compute gets to the physical world, the more constrained and specialised it becomes.

Tier	Typical hardware	Available memory	What it can host
Constrained device edge	Fixed-function sensors and actuators, PLCs, RTUs, engine control units	KBs to low MBs	Embedded code tightly coupled to the hardware; TinyML with specialised toolsets
Smart device edge	Arm and x86 gateways, embedded PCs, hubs, routers, hardened headless servers	More than 256 MB, still constrained	Containers or VMs; accelerators (GPU, FPGA, TPU) added as needed
Service provider and edge data centres	Standard servers and networking with accelerators, including MEC hosts at base stations or regional sites	Server class	Cloud-native workloads orchestrated with tools such as Kubernetes
Centralised cloud	Large data centres	Practically unlimited	Training, fleet-wide configuration and management

Cameras do not sit neatly in one row. LF Edge notes that high-resolution video can be processed inside a smart camera acting as the edge node or on a nearby edge server. In both cases only flagged events or clips go to the cloud. LF Edge itself treats the boundaries as fluid, but argues that technical and logistical limits still decide where a workload is best run.

How is an IoT edge device different from a gateway or an edge server?

The practical dividing line is memory, because memory decides which software model the device can run. LF Edge puts the practical lower limit for hosting virtualised or containerised workloads at roughly 256 MB. Below that, software has to be embedded and tightly coupled to the hardware. An IoT edge device in the narrow sense (a sensor, a PLC, a microcontroller board) lives below that line. A gateway or hardened small server lives above it and can host virtualised or containerised workloads.

Microsoft’s Azure IoT Edge documentation describes a gateway that connects downstream devices to IoT Hub. It shields them from the internet by sitting between an operational technology network with no connectivity and an IT network that has it. It can also multiplex all those devices over one cloud connection, which requires AMQP as the cloud-bound protocol. The same documentation lists local analytics as a benefit of every gateway pattern: AI services process downstream data on site, and only a subset is sent upward. For the wider picture of IoT at the edge and why teams put compute there, that article covers the benefits side.

An edge server is a small-scale server resource placed near the network edge. In 5G multi-access edge computing, the host provides virtualised compute, storage and networking to run edge applications. Because those hosts have fewer resources than the cloud, a 2023 survey by Nencioni, Garroppo and Olimid argues that heavyweight hypervisors fit poorly and that containers have gained ground because they are easy to build, migrate and remove. One study the survey cites measured a container boot time of 0.623 s against 32 s for a VM, although the VM gave higher HTTP throughput in that test.

Which accelerator should run a vision model at the edge?

The published edge benchmarks and vendor specifications cover four accelerator families. The first is embedded GPU modules such as NVIDIA Jetson. The second is ASICs such as Google’s Edge TPU. The third is vision processing units such as Intel’s Movidius Myriad X, used in the Neural Compute Stick 2. The fourth is NPU modules such as Hailo-8.

The vendors’ own specifications give these headline figures:

• Google’s Coral datasheet gives each Edge TPU 4 TOPS at 2 W, with an operating range of -40 to +85 °C for the dual M.2 module.
• Hailo states up to 26 TOPS for Hailo-8 at a typical 2.5 W, with an industrial grade rated -40 to 85 °C.
• Intel’s Myriad X brief claims over 1 TOPS on deep neural network inference from the Neural Compute Engine, and direct connection of up to 8 HD cameras over MIPI lanes.
• NVIDIA’s technical brief gives Jetson AGX Orin up to 275 TOPS, with power configurable between 15 and 60 W.

We would not compare devices on these headline figures. Peak TOPS are quoted at different precisions and sparsity assumptions, and none of them tells you how your model’s operators map onto the chip.

Measured comparisons show devices trading accuracy, latency and power against one another. A 2023 facial-expression-recognition study on arXiv measured a Raspberry Pi alone at 4.70 ms and 1.41 W, a Jetson Nano in high-power mode at 1.93 ms and 2.37 W, a Pi with a Coral USB TPU at 1.87 ms and 0.82 W, and a Coral Dev Board at 0.47 ms and 0.55 W. The Pi and both Coral setups reached 98.88% accuracy against 95.45% on the Jetson Nano, and the authors attribute the differences to each device running a different precision (Float32, Float16 or Int8). A 2025 sustainability study on arXiv tested a different set of devices and workloads: the Jetson Nano with TensorRT gave the fastest inference and the lowest memory use, and the Raspberry Pi was slowest and used the most memory.

Which device comes out ahead depends on the model and the device. A 2024 object-detection benchmark found TPU-equipped Raspberry Pi 4 and 5 boards best for SSD models, and the Jetson Orin Nano best for EfficientDet Lite and YOLOv8. We read that as a result tied to those tests, not as a ranking that carries over to your network.

Precision support constrains which models a device can run. Coral’s Edge TPU only runs models converted to TensorFlow Lite and quantised to 8-bit integers. In the 2024 benchmark, YOLOv8 had to drop from 640×640 to 320×320 input to fit TPU deployment, and the TPU configurations lowered YOLOv8 accuracy. The authors attribute accuracy differences across hardware mainly to precision, for example INT8-only execution on the TPU boards against FP16 on the Orin Nano. For a head-to-head on specific modules, our comparison of Jetson, Coral, Hailo and OAK-D for embedded CV deployment goes device by device.

What makes hardware edge-ready?

Edge-ready hardware has to keep running at sites with no IT staff on hand. ZPE Systems lists seven requirements. They work as a checklist:

1. Compute matched to the application: processor, memory, storage and any coprocessor sized to the workload, not to a generic spec.
2. Small, ruggedised chassis: it has to fit roadside cabinets or machine enclosures and tolerate vibration or humidity.
3. Power: standard PDUs and UPSes where they exist, and energy-efficient designs on batteries or solar where they do not.
4. Wired and wireless connectivity: cellular 4G/5G adds flexibility and ideally provides failover.
5. Out-of-band management: a path to the device that does not depend on production infrastructure, so an outage does not mean a truck roll.
6. Security: Zero Trust controls and a hardware root of trust for cryptographic functions and key management.
7. Visibility: the ability to monitor what the fleet is doing.

LF Edge adds two further requirements for enterprise and industrial edge hardware. One is extreme temperature support with fanless design, plus certifications such as Class 1 / Division 2. The other is service life: industrial edge assets need 5 to 7+ years, against 12 to 18 months for typical consumer devices. At the server end, Dell’s rugged PowerEdge XR line is built for environments outside traditional data centres. It is specified for -5 to 55 °C (up to -20 to 65 °C on select XR8000 configurations), shock, dust and vibration, and carries up to three 70 W NVIDIA L4 GPUs in some models. It also ships with iDRAC remote management and a silicon root of trust.

Thermal limits throttle edge modules

NVIDIA’s thermal design guide states that the Jetson Orin NX and Orin Nano modules come without a system-level thermal solution. The system designer is responsible for keeping components below their limits. The guide gives a maximum operating SoC temperature of 99 °C and a shutdown temperature of 105 °C. When software thermal management cannot hold the temperature, hardware clock throttling engages as a last resort. That throttling “significantly” reduces performance, and its settings cannot be altered.

An under-cooled module in a hot enclosure is therefore throttled, at a significant cost in performance, to prevent shutdown. Coral’s datasheet makes the same point for its accelerator: heat scales with computational load, and the Edge TPU junction must stay below 115 °C.

Edge computing vs cloud computing: where each workload belongs

A request’s response time is the sum of network round trip, queuing delay and service time. A 2026 arXiv modelling study shows that, on identical server hardware, a distributed edge only gives faster responses when its lower network delay more than offsets the higher queuing delay of its smaller, separate queues. The same study cites modelling and experiments showing the edge beats the cloud at low utilisation. At medium to high utilisation, though, queuing at the edge can push latency orders of magnitude above simply sending requests to the cloud.

The capacity penalty follows from pooling. In the paper’s model, k edge sites each receive Poisson load λ. Provisioning each site for its 95th percentile needs λ + 2√λ per site, so kλ + 2k√λ in total. The cloud, which sees the combined load, needs only kλ + 2√(kλ). Since √(kλ) is smaller than k√λ for any k above 1, the edge has to buy more capacity to serve the same peak.

Several smaller caveats point the same way:

• Cost: edge sites lose economies of scale and add construction, installation, maintenance and management costs.
• Workload skew: user mobility leaves some edge sites busier than others, while a central cloud multiplexes bursty load across many more servers.
• Mobility: stateful migrations such as live video between edge nodes add significant overhead.
• Burstiness: edge clouds suit applications whose request rates vary sharply over short periods less well.
• Provisioning: over-provisioning edge capacity wastes investment, and under-provisioning risks SLA violations.

Edge wins when network delay or bandwidth is the problem. In vision, that means video too heavy to backhaul, so only flagged events leave the site, and time-critical detection such as stopping a production line on a flaw. Shi et al. (IEEE IoT Journal, 2016) cite a face-recognition prototype whose response time fell from 900 to 169 ms when computation moved from the cloud to the edge. The common split, in LF Edge’s description, is to train where resources are plentiful and push the models out to the edge for inference. For a workload that tolerates around 30 to 100 ms, LF Edge places it within reach of regional edge data centres instead of on-site hardware. In our view, that is the first question to settle before buying any box. We cover where the split falls for vision pipelines in computer vision and cloud computing: where the workload actually splits.

How do I size edge hardware for an inference workload?

Size the hardware against the deployed model and the site:

1. Fix the model family and input resolution. Inference time scales with model complexity, and larger images raise the number of convolution operations and the energy each inference uses.
2. Check precision and toolchain per device. Each platform has its own stack (LiteRT on Raspberry Pi, TensorRT on Jetson, OpenVINO on Intel, EdgeTPU on Coral), and this fragments deployment. A device that requires INT8 rules out models that lose too much accuracy when quantised.
3. Set batch size by device. For ResNet-50 and MobileSSD, the 2025 arXiv study found batch 1 the only viable option on its Raspberry Pi, 4 to 8 on the Intel NCS, 8 to 16 on Coral and 16 to 32 on Jetson Nano.
4. Check memory headroom. Larger batches can saturate memory on constrained devices, and the 256 MB line decides whether containers are an option at all.
5. Size the thermal solution against the worst ambient temperature, using the module’s steady-state power under the target workload.
6. Leave utilisation headroom without over-provisioning. Edge latency degrades as utilisation rises, and idle edge capacity is wasted investment.

Worked example: heat-sink sizing for a Jetson Orin module at 55 °C ambient

NVIDIA’s thermal design guide works the calculation for a 55 °C ambient and a 25 W total module power. The SoC target is 95 °C, which leaves 4 °C of headroom below the 99 °C operating limit for sensor inaccuracy and workload swings. The required heat-sink performance is (95 − 55) / 25 = 1.60 °C/W. The guide puts sensor accuracy at ±3 °C, so the headroom you choose reflects how much throttling you can tolerate.

The deployment target, not the model, is usually what breaks

Our position is that production AI deployment frequently fails on runtime, hardware-target and platform constraints rather than on model quality. A model that performs in training is a different artefact from one that performs on the target, and the gap between them is engineering work. The researchers behind the 2025 sustainability study describe it this way: finding a working implementation for each model and learning the conversion path for Jetson and the Neural Compute Stick took considerable time and effort. Shi et al. made the general point in 2016: edge nodes are heterogeneous, their runtimes differ, and writing one application that deploys across them is hard.

This is why we would choose edge hardware against a named constraint: the latency the line needs, the enclosure’s thermal ceiling, the precision the accelerator supports. Once the constraint is named, porting work is scoped to reach it, not done for its own sake. TechnoLynx’s AI engineering for IoT and embedded devices practice covers porting models onto edge boards, embedded targets and novel silicon. For the step-by-step mechanics of conversion and runtime choice, see how to deploy computer vision models on edge devices.