Cloudy: heterogeneous middleware for in time queries processing

Abstract

Parallel share-nothing architectures are currently used to handle large amounts of data arriving in real-time for processing. The continuous increase on data volume and organization, introduce several limitations to scalability and quality of service (QoS) due to processing requirements and joins. Parallelism may improve query performance, however some business require timely results (results not faster or slower than specified) which, even with additional parallelism and significant upgrade costs (both monetary and due to disturbance of normal operations), cannot be guaranteed. We propose a timely-aware execution architecture, Cloudy, which balances data and queries processing among an elastic set of non-dedicated and heterogeneous nodes in order to provide scale-out performance and timely results, nor faster or slower, using both Complex Event Processing (CEP) and database (DB). Data is distributed by nodes accordingly with their hardware characteristics, then a set of layered mechanisms rearrange queries in order to provide in timely results. We present experimental evaluation of Cloudy and demonstrate its ability to provide timely results.