Increasing Convolutional Neural Networks Training Speed by Incremental Complexity Learning

Abstract

Convolutional Neural Networks have been successfully applied in several image related tasks. On another hand, there are some overhead costs in most of the real applications. Often, the Deep Learning techniques demand a huge amount of data for training and also a crescent need for handling high definition images. For this reason, late network architectures are getting even more complex and deeper. These factors lead to a long training time even when specific hardware is available. In this paper, we present a novel incremental training procedure which is able to train faster with small performance losses, based on measuring and ordering the relative complexity of subsets of the training set. The findings reveal an expressive reduction in the number of training steps, without critical performance losses. Experiments showed that the proposed method can be about 40% faster, with less than 10% of accuracy loss.