Improving Network Processing Concurrency using TCPServers

Exponentially growing bandwidth requirements and slowing gains in processor speeds have led to the popularity of multiprocessor architectures. Network stack parallelism is increasingly important to support such architectures. In this paper, we present techniques to improve network stack concurrency using our previous work, TCPServers, a system architecture for offloading network processing within an SMP system. TCPServers dedicates a subset of processors as packet processing engines (PPEs), which handle all asynchronous network events and perform receive processing. We introduce Receive Queues, data structures associated with each socket that store incoming network packets and are accessed exclusively at the PPEs. Using Receive Queues, we modify TCPServers based network stacks to incorporate early packet demultiplexing. We also present an efficient proportional fair scheduling algorithm, which processes incoming packets at the priority of the destination socket. Our experimental evaluation demonstrates that our modifications reduce the scheduling and synchronization overheads and improve the aggregate TCP/IP throughput by up to 75% compared against the default SMP stack. We also show that our system sustains this throughput, even when a large number of short lived connections are present.