EB-CNN: Ensemble of branch convolutional neural network for image classification

Abstract

Traditionally, image classifiers using Convolutional Neural Networks (CNNs) have all their outputs combined into a single layer. This assumes all categories are equally distinct and independent. However, some classes are harder to distinguish by using just this single output layer for classification due less flexibility of the model to learn complex relationships and representations within the data. Different classes may require different levels of abstraction or representation, which cannot be adequately captured by a single output layer. This paper proposes an ensemble method that combine different layers or branches of CNN network. The approach divides the CNN network i.e. VGG16 into five different distinct branches to simulate the coarse, intermediate and fine spatial scale corresponding to the hierarchical structure of the deep learning network. However, a possible problem with combining all branch models to create a dense pool of candidate for ensemble learning is that the potential lack of diversity among the classifier models, which can hinder the ensemble’s ability to generalize and may lead to suboptimal performance. Therefore, in order to improve the predictive performance, we designed a heuristic ensemble selection method that chooses the relevant models from the pool of saved models based on the their accuracy. We have performed experiments on 6 different datasets. The results show that our approach outperforms the baseline CNN model that rely on the single layer for making a final decision.