A Proxy-Based Query Aggregation Method for Distributed Key-Value Stores

Abstract

Distributed key-value stores (D-KVS) are critical backbone for SNS and cloud services. Some D-KVS are based on a ring architecture with multiple database nodes to handle large amount of data. Any of them can receive queries from clients, and the node forwards queries to an adequate node if necessary. Therefore, this architecture causes heavy overhead of packet processing for each node.Some D-KVS have adopted fast packet processing frameworks like DPDK, but this is not enough to handle huge amount of requests. We introduce a query aggregation method to D-KVS to reduce the network traffic. In our approach, client queries are aggregated into a few large-sized query packets by a centralized proxy. The proxy receives every query from the clients, and it routes aggregated queries to the destination nodes. The proxy is built on top of DPDK-based network stack and can deal with the growing of the clients by increasing the number of CPU cores for packet handling. We evaluated with the environment of three Cassandra nodes linked with 10 Gbps network. Our approach improved throughput by 19% compared with the non-proxy Cassandra.