The Analysis and Optimization of CNN Hyperparameters with Fuzzy Tree Model for Image Classification

Abstract

The meaningful performance of convolutional neural network (CNN) has enabled the solution of various state-of-the-art problems. Although CNNs achieve satisfactory results in computer-vision problems, they still have some difficulties. As the designed CNN models are deepened to achieve much better accuracy, computational cost and complexity increase. It is significant to train CNNs with suitable topology and training hyperparameters that include initial learning rate, minibatch size, epoch number, filter size, number of filters, etc. because the initialization of hyperparameters affects classification results. On the other hand, it is not possible to make a definite inference for the hyperparameter initialization and there is uncertainty. This study is carried out to model uncertainty using fuzzy inference system (FIS). The designed fuzzy model provides estimation of classification result depending on CNN topology and training hyperparameters. GoogleNet and Inceptionv3 that contain inception-modules, ShuffleNet that contains shuffleblocks, DenseNet201 that contains dense-blocks, EfficientNet, ResNet18, ResNet50, ResNet101, and MobileNetv2 that contain residual-blocks, and InceptionResNetv2 that includes both inception-modules and residual-blocks were evaluated as CNN models. Test sample dataset was obtained by training CNN models with various training hyperparameter combinations. CNN models were trained on Animal Diagnostics Lab (ADL) which is a histopathological dataset includes healthy and inflamed kidney, lung, and spleen images. A new FIS tree model that is more computationally efficient and easier to understand than a single FIS was designed and classification accuracy prediction of CNN models depending on hyperparameter combinations was performed. The best, the worst, and the average classification accuracies obtained with CNN models that use best training hyperparameter set are 97.70%, 93.60%, and 96.30%, respectively. Moreover, Cifar10 and Cifar100 benchmark datasets were experimented to reveal true capability and limitations of the proposed approach. Experimental results indicate that the designed FIS tree model provides a successful hyperparameter evaluation mechanism with an average RMSE value of 1.2652.