FGLB: A fine-grained hardware intra-server load balancer based on 100 G FPGA SmartNIC

Abstract

In today's data centers, workloads including multiple services and requests are processed in parallel within a server with many CPU cores. Therefore, meeting the intra-server load balancing is very important to improve the utilization of CPU resources in the data centers. However, the existing methods cannot well meet the intra-server load balancing in high-throughput scenarios. The software-based methods generally utilize CPU cores to parse and dispatch packets. They work well at low throughput, but they have high CPU overhead at high throughput, leading to packet loss and high latency issues. The hardware-based methods parse the packet and compute a hash over its metadata in hardware and perform load balancing in a coarse-grained manner based on the hash value. They have the ability to work at high throughput with the advantage of low overhead but are less well in balance effect and flexibility. We, therefore, propose an intra-server load balancer based on the reconfigurable hardware, FPGA, to meet the requirements for load balancing within servers in high-speed application scenarios. Our method improves the load-balancing gran- ularity of hardware-based method. It not only has high throughput but also has a good balance effect and flexibility. We implemented and evaluated our method on a 100 G FPGA SmartNIC. The evaluation result shows our method can reduce the load imbalance ratio by an order of magnitude when the distribution of flow size is uneven compared to the current widely used hardware-based method.