Spatiotemporal crowds features extraction of infrared images using neural network

Abstract

Crowds can lead up to severe disasterous consequences resulting in fatalities. Videos obtained through public cameras or captured by drones flying overhead can be processed with artificial intelligence-based crowd analysis systems. Being a hot area of research over the past few years, the goal is not only to identify the presence of crowds but also to predict the probability of crowd-formation in order to issue timely warnings and preventive measures. Such systems will significantly reduce the probablity of the potential disasters. Developing effective systems is a challenging task, especially due to factors such as naturally occuring diverse conditions, variations in people or background pixel areas, noise, behaviors of individuals, relative amounts/distributions/directions of crowd movements, and crowd building reasons. This paper proposes an infrared video processing system based on U-Net convolutional neural network for crowd monitoring in infrared video frames to help estimate the people crowd with normal or abnormal trends. The proposed U-Net architecture aims to efficiently extract crowd features, achieve sufficient people marking-up accuracy, competitively with optimal network configurations in terms of the depth and number of filters to consequently minimise the number of coefficients. For further faster processing, hardware resources/implementation area savings, and lower power, the optimized network coefficients measured are represented in Canonic-Signed Digit with minimal number of nonzero (± 1) digits, minimizing the number of underlying shift-add/subtract operations of all multipliers. The achieved significantly reduced computational cost makes the proposed U-Net effectively suitable for resource-constrained and low power applications.