Configuring and Coordinating End-to-End QoS for Emerging Storage Infrastructure

Modern data center storage systems are invariably networked to allow for consolidation and flexible management of storage. They also include high performance storage devices based on flash or other emerging technologies, generally accessed through low-latency and high throughput protocols such as NVMe (or its derivatives) carried over the network. With the increasing complexity and data-centric nature of the applications, properly configuring the quality of service (QoS) for the storage path has become crucial for ensuring the desired application performance. Such QoS is substantially influenced by the QoS in the network path, in the access protocol, and in the storage device. In this paper, we define a new transport level QoS mechanism for the network segment and demonstrate how it can augment and coordinate with the access level QoS mechanism defined for NVMe, and a similar QoS mechanism configured in the device. We show that the transport QoS mechanism not only provides the desired QoS to different classes of storage accesses but is also able to protect the access to the shared persistent memory (PM) devices located along with the storage but requiring much lower latency than storage. We demonstrate that a proper coordinated configuration of the 3 QoS’es on the path is crucial to achieve the desired differentiation depending on where the bottlenecks appear.