RIFL: A Reliable Link Layer Network Protocol for FPGA-to-FPGA Communication

Abstract

More and more latency-sensitive applications are being introduced into the data center. Performance of such applications can be limited by the high latency of the network interconnect. Because the conventional network stack is designed not only for LAN, but also for WAN, it carries a great amount of redundancy that is not required in a data center network. This paper introduces the concept of a three-layer protocol stack that can replace the conventional network stack and fulfill the exact demands of data center network communications. The detailed design and implementation of the first layer of the stack, which we call RIFL, is presented. A novel low latency in-band hop-by-hop re-transmission protocol is proposed and adopted in RIFL, which guarantees lossless transmission for links whose longest wire segment is no more than 150 meters. Experimental results show that RIFL achieves 218 nanoseconds round-trip latency on 3 meter zero-hop links, at a throughput of 104.7 Gbps. RIFL is a multi-lane protocol with scalable throughput from 500 Mbps to above 200 Gbps. It is portable to most of the recent FPGAs. It can be the enabler of low latency, high throughput, flexible, scalable, and lossless data center networks.