Homogenous Ensembles of Neuro-Fuzzy Classifiers using Hyperparameter Tuning for Medical Data

Abstract

Neuro-fuzzy techniques have been widely used in many applications due to their ability to generate interpretable fuzzy rules. Ensemble learning, on the other hand, is an emerging paradigm in artificial intelligence used to improve performance results by combining multiple single learners. This paper aims to develop and evaluate a set of homogeneous ensembles over four medical datasets using hyperparameter tuning of four neuro-fuzzy systems: adaptive neuro-fuzzy inference system (ANFIS), Dynamic evolving neuro-fuzzy system (DENFIS), Hybrid fuzzy inference system (HyFIS), and neuro-fuzzy classifier (NEFCLASS). To address the interpretability challenges and to reduce the complexity of high-dimensional data, the information gain filter was used to identify the most relevant features. After that, the performance of the neuro-fuzzy single learners and ensembles was evaluated using four performance metrics: accuracy, precision, recall, and f1 score. To decide which single learners/ensembles perform better, the Scott-Knott and Borda count techniques were used. The Scott-Knott first groups the models based on the accuracy to find the classifiers appearing in the best cluster, while the Borda count ranks the models based on all the four performance metrics without favoring any of the metrics. Results showed that: (1) The number of the combined single learners positively impacts the performance of the ensembles, (2) Single neuro-fuzzy classifiers demonstrate better or similar performance to the ensembles, but the ensembles still provide better stability of predictions, and (3) Among the ensembles of different models, ANFIS provided the best ensemble results.