Visual Saliency Modeling with Deep Learning: A Comprehensive Review

Visual saliency models mimic the human visual system to gaze towards fixed pixel positions and capture the most conspicuous regions in the scene. They have proved their efficacy in several computer vision applications. This paper provides a comprehensive review of the recent advances in eye fixation prediction and salient object detection, harnessing deep learning. It also provides an overview on multi-modal saliency prediction that considers audio in dynamic scenes. The underlying network structure and loss function for each model are explored to realise how saliency models work. The survey also investigates the inclusion of specific low-level priors in deep learning-based saliency models. The public datasets and evaluation metrics are succinctly introduced. The paper also makes a discussion on the key issues in saliency modeling along with some open problems and growing research directions in the field.