Cluster-Boosted Artificial Neural Networks: Theory, implementation, and performance evaluation

Abstract

This study introduces a new clustering technique to boost Artificial Neural Networks’ (ANNs’) performance. The term “Cluster-Boosted Artificial Neural Networks” (“CBANNs”) is coined for ANNs using this technique. By adding cluster identifiers as extra input features, CBANNs enhance conventional ANNs and improve the model’s ability to identify underlying patterns in complicated data landscapes. This method offers a solution to some limitations of standard ANNs, which often struggle with high-dimensional data, local minima, and nonlinear relationships. Without the need for manual feature engineering or in-depth domain knowledge, CBANNs greatly increase prediction accuracy by employing unsupervised clustering, using k-medoids, to build a more structured input space. Various numerical results are presented which validate the superior predictive ability of CBANNs across nine benchmark functions, including De Jong’s 5th, Griewank, and Rastrigin functions. Compared to conventional ANNs with identical hyperparameters, CBANNs achieve error reductions of up to 98%, consistently demonstrating higher performance on functions with intricate geometries and multiple minima. Furthermore, CBANNs are applied to a terrain modeling problem, which proved that CBANNs can reduce the prediction error by up to 95% compared to standard ANNs, indicating their potential for high-precision applications. These findings underscore the CBANN’s ability to generalize effectively in challenging datasets, suggesting its broader applicability in fields that demand accuracy in the presence of complex data distributions.