Compressed Hierarchical Bitmaps for Efficiently Processing Different Query Workloads

Abstract

Today the amount of data that is being processed is growing manyfold. Fast and scalable data processing systems are the need of the hour because of the data deluge. Indexing is a very common mechanism used in data processing systems for fast and efficient search of the data. In many systems, the I/O needed to read and fetch the relevant part of the index into the main memory dominates the overall query processing cost. My research is focused on reducing this I/O cost by effective indexing algorithms. I have particularly focused on using bitmap indices, which are a very efficient indexing mechanism particularly used in data warehouse environments due to their high compressibility and ability to perform bitwise operations even on compressed bitmaps. Column-store architecture is preferred in such environments because of their ability to leverage bitmap indices. Column domains are often hierarchical in nature, and hence using hierarchical bitmap indices is often beneficial. I have designed algorithms for choosing a subset of these hierarchical bitmap indices for 1D as well as spatial data in order to execute range query workloads for various different scenarios. I have shown experimentally that these solutions are very efficient and scalable. Currently, I am focusing on leveraging hierarchical bitmap indices to solve approximate nearest neighbor queries.