Modern data-center accelerators depend on high-performance network connectivity for disaggregation and distributed computation. In addition, data centers require inline acceleration of packets processing for network functions. These seemingly unrelated requirements share a single challenge: designing efficient interfaces between the accelerator and the network.

The simplest solution is to rely on existing software infrastructure for networking. CPU software can mediate between an accelerator and an ASIC NIC. The network stack can take advantage of a full range of NIC offloads in such systems. Unfortunately, this solution puts an extra load on the CPU, processing every accelerator message.

An accelerator-hosted design integrates networking functionality within the accelerator. It is a common solution for FPGA-based SmartNICs and network-attached accelerators. Such solutions post a trade-off to designers, who must allocate accelerator area and resources between networking functionality and the desired acceleration.

Bump-in-the-wire designs, such as those used in NICA, place an ASIC NIC between the accelerator and the host. They reuse some ASIC NIC functionality and the ASIC NIC’s software stack. Nevertheless, as NICs typically expose their full features only through their PCIe interface, this design limits the offloads the accelerator can utilize.

FlexDriver is a new system design that provides accelerators with direct control over commodity NICs through their PCIe interface. This design allows accelerators to build upon existing NIC offloads such as virtualization, tunneling, or RDMA, reducing the area wasted on networking within the accelerator.

Challenge: on-chip memory utilization

One challenge of our approach is the amount of memory required for operating the NIC. A software NIC driver allocates memory for descriptor rings, packet buffers, completion queues, etc. We address it by keeping a compressed representation of the data-structures on-chip, translating to the NIC’s proprietary interface on the fly.

Example applications

We built three example accelerators for FlexDriver:

• A disaggregated LTE cipher uses the NIC’s RDMA transport for reliable communication.
• An IP-defragmentation accelerator adds a custom inline acceleration stage between VXLAN tunnel decapsulation and receive-side scaling offloads in the NIC.
• An IoT cryptographic-token authentication accelerator demonstrates relying on the NIC’s traffic shaping capabilities for allocating computational resources.

Acknowledgments

This project is a collaboration with NVIDIA Networking.

Sources

• The analytical analysis of driver memory usage and expected PCIe overheads are available.
• The LTE cipher and IoT token authentication accelerators are open-source.
• For further details, see our paper FlexDriver: A Network Driver for Your Accelerator, ASPLOS 2022.