AARS: A novel adaptive archive-based efficient counting method for machine learning applications

Abstract

For many machine learning methods, while dealing with problems such as classification, clustering, prediction, and association rule mining, counting the occurrences of given queries plays a crucial role. However, these methods, which usually function in two different steps, i.e., learning and sampling, become impractical for large datasets due to computational costs or excessive memory consumption. Therefore, this paper proposes a novel approach to handle the counting queries. The proposed method is an adaptive archive-based method that offers efficient archiving with reduced computational time and moderate mem-ory requirements. We conduct numerous experiments to show the performance and scalability of the proposed approach on random queries, learning probabilistic networks, and association rule mining. From experimental results, we see that our proposed method outperforms the previously proposed ADtree, Bitmap and Radix strategies when applied to the datasets with higher dimensions and a large set of observations.