Improving multi-class classification: scaled extensions of harmonic mean-based adaptive k-nearest neighbors

Abstract

This paper proposes a novel extension of the harmonic mean-based adaptive k-nearest neighbors (HMAKNN) algorithm, called scaled HMAKNN (SHMAKNN), which builds on HMAKNN’s strengths to achieve improved multi-class classification accuracy. HMAKNN uses a modified voting mechanism based on the harmonic mean and adaptive k-value selection to address issues like the sensitivity to k-value selection and the limitations of majority voting. SHMAKNN further improves the decision process by adjusting the components of the harmonic mean, focusing on voting values and the average distances of each class label. Additionally, SHMAKNN applies a re-scaling process to adjust the distances of the nearest neighbors within a specific range, enhancing the consistency of distances at different scales. These improvements help align the elements of the harmonic mean more effectively, leading to a balanced and less biased classification process. The study utilized 26 benchmark datasets, carefully curated to ensure accuracy and consistency, selected from diverse domains to evaluate the proposed method on real-world problems. These datasets were chosen to represent challenges like noise, imbalance, and sparsity, ensuring robustness in handling common data complexities. Additionally, small to medium-sized datasets were used to reduce computational burden and allow for efficient evaluation. The evaluation results show that the proposed SHMAKNN models outperform existing methods in both accuracy and F1-score for datasets with four or more classes. Specifically, SHMAKNN achieved the highest average accuracy and F1-score (86.36% and 86.16%) compared to HMAKNN (86.10% and 85.74%) and traditional k-nearest neighbors (84.87% and 84.69%). The performance improvements were validated using Friedman’s test at a significance level of 0.05, confirming their statistical significance of the results. Consequently, the findings indicate that the proposed algorithm exhibits remarkable performance, thereby confirming its reliability and validity in the context of real-world applications, particularly those involving multiple classes.