GPU accelerated video super resolution using transformed spatio-temporal exemplars

Abstract

Super-resolution (SR) is the method of obtaining high-resolution (HR) images or image sequences from one or more low-resolution (LR) images of a scene. Huang et al. in 2015 proposed a transformed self-exemplar internal database technique which takes advantage of fractal nature in an image by expanding patch search space using geometric variations. This method fails if there is no patch redundancy within and across image scales and also if there is a failure in detecting vanishing points (VP) which are used to determine perspective transformation between LR image and its sub-sampled form. In this paper, we expand the patch search space by taking advantage of temporal dimension of image frames in the scene video and also use an efficient vanishing point (VP) detection technique by Lezama et al. in 2014 and are thereby able to successfully super-resolve even the failure cases of Huang et al. and an overall improvement in PSNR. We also focused on reducing the computation time by exploiting the embarrassingly parallel nature of the algorithm. We achieved a speedup of six on multi-core, up to 11 on GPU, around 16 on hybrid platform of multi-core and GPU by parallelising the proposed algorithm. Using our hybrid implementation, we achieved 32x super-resolution factor in limited time.