Optimising Resource Use Through Low-Precision Feature Selection: A Performance Analysis of Logarithmic Division and Stochastic Rounding

Abstract

The growth in the number of wearable devices has increased the amount of data produced daily. Simultaneously, the limitations of such devices has also led to a growing interest in the implementation of machine learning algorithms with low-precision computation. We propose green and efficient modifications of state-of-the-art feature selection methods based on information theory and fixed-point representation. We tested two potential improvements: stochastic rounding to prevent information loss, and logarithmic division to improve computational and energy efficiency. Experiments with several datasets showed comparable results to baseline methods, with minimal information loss in both feature selection and subsequent classification steps. Our low-precision approach proved viable even for complex datasets like microarrays, making it suitable for energy-efficient internet-of-things (IoT) devices. While further investigation into stochastic rounding did not yield significant improvements, the use of logarithmic division for probability approximation showed promising results without compromising classification performance. Our findings offer valuable insights into resource-efficient feature selection that contribute to IoT device performance and sustainability.