Skeleton Based Human Activity Prediction in Gait Thermal images using Siamese Networks

Abstract

Thermal image is formed by capturing of radiation emitted by object to its surroundings and the difference in radiation of object and its surroundings. The advantages of Thermal images over Normal RGB images is the ability to visible at night time irrespective of illumination conditions and weather conditions like rain, fog, mist, and dust. Thermal images can form images in typical situations like smoke, dust, and high intensity, where the normal RGB camera fails to capture image. Human Activity Recognition in Thermal Images is still a challenging task due to less availability of Thermal Human Activity Datasets. This research work has proposed a human activity recognition system using Siamese Networks of Gait Skeleton Thermal Images. The proposed approach can train a new human activity by extracting Gait Skeleton from existing RGB videos and can be compared to a gait skeleton extracted from a Thermal video in case of utilizing very less thermal videos for human activity recognition. Thermal videos are extracted from IITR- IAR dataset and the performance is analyzed with CNN+LSTM, LRCN, Inflated 3D CNN, Siamese using accuracy and the proposed model has achieved a better accuracy when compared to CNN+LSTM, LRCN, Inflated 3D CNN.