Efficient dequeuing technique for distributed messaging systems processing massive message volumes

Abstract

In the Internet of Things (IoT) era, message queue systems are highly required to control the large amount of message traffic between field devices, heterogeneous back-end applications, and services with high interoperability. In today's trend of rapid development cycle, the role and number of each back-end application system (BES) are frequently modified. In this case, the message queue systems receiving dequeue requests from BESs cannot maintain their preliminarily estimated maximum throughput, because the proportional increase of missed dequeues (i.e. no message is available) results in a waste of computational resources. Therefore, we propose the retry dequeue-request scheduling (RDS) method, which decreases the number of dequeue requests from BESs by delaying the replies to BESs when missed dequeues occur. Evaluation results show that the RDS method gives a 70% throughput improvement compared with the conventional method.