Automatic Depression Detection Based on Merged Convolutional Neural Networks using Facial Features

Clinical depression is one of the crucial medical conditions affecting a substantial portion of today’s world population. This paper proposes a novel technology based on deep learning tools for efficient detection of clinical depression in potential patients. A novel algorithm is proposed to label a recent AFEW-VA dataset in terms of creating the depressed class and the non-depressed class based on the valence and arousal values for various individuals from their video frames. Furthermore, the full facial regions, the eye regions, and the mouth regions from the classified dataset are extracted as the regions of interest (ROIs) to be utilized to train three different pre-trained 2DCNN models, namely, ResNet50, VGG16, and InceptionV3 by using transfer learning. For each 2D-CNN architecture, a novel algorithm is proposed to merge the models trained on the three ROIs. It is observed that the merged model, combining all the three ROIs outperforms the individual models or a merged model merging only two of the three ROIs in terms of obtaining a higher accuracy of depression detection. It is also observed that the merged models based on the ResNet50 architecture results in the best accuracy value of 0.95 as compared to the VGG16 and InceptionV3 architectures.