As network speeds increased from 1G to 10G to 112G SerDes and Ethernet speeds from 25G to 100G to 200G/400G and, now, 800G, the thinking about hardware architecture among hyperscale data center experts has shifted. A peek inside a traditional data center architecture would reveal CPUs, memory, storage, and network components. In recent years, a consensus has formed around the thought that general-purpose CPUs are no longer the best place to run infrastructure functions. A lot of overhead is required to support functions like hypervisors, routing, and load balancing, as well as IO-intensive security functions like deep packet inspection and data storage encryption/decryption. Some hyperscalers have estimated that around half of CPU cycles are consumed by non-revenue-generating infrastructure tasks—a key consideration for those who make their business selling compute cycles.
SmartNICs can take on much of the heavy lifting, freeing CPUs to focus on revenue-generating application processing. The intelligence of SmartNICs comes from their programmability, along with their hardware acceleration capabilities. Bringing together wired networking and computational resources on a single card, SmartNICs feature their own on-board processor, accelerators with a custom ASIC implementation, or an FPGA and high-speed memory and IOs.
Integrating intelligence into these networking components has evolved to accommodate changes in data center bandwidth requirements. Initially, there was a focus on offloading infrastructure functions away from the host server CPU. Then, platform functions like cloud technologies were offloaded, from hypervisors to virtual machines, containers, and then microservices. Now, we’re seeing demands for application acceleration; for example, the speeding up of tasks such as video transcoding, encryption/decryption, and packet processing. CPUs can’t support the packet processing demands at today’s highest line rates, so offloading to programmable hardware, in the form of SmartNICs, makes sense.