A method for automatic recognition of deviant behavior of people based on the integration of computer vision and knowledge management technologies to support decision-making by operators of video monitoring systems

Ensuring safety of people in the urban environment is an important area for improving the quality of people’s lives, and the development of modern intelligent technologies creates new opportunities to achieve this goal. The use of modern intelligent technologies can realize the potential of video surveillance systems, making possible the automatic recognition of dangerous situations in real time in order to take timely measures to handle them and provide aid to victims. Often, a dangerous situation is caused by deviant behavior of people – robbery, fight, vandalism, etc. But the existing works focused on recognizing deviant behavior are only focused on recognizing short-term distinguishing features, such as punches, falls or weapons in the hands of a person. At the same time, such features may often be absent, for instance, when a passerby is robbed but no fighting occurs, which is why the task of recognizing complex long-term scenes of deviant behavior remains unresolved. This paper proposes a method for automatic recognition of long-term human deviant behavior scenes, characterized by the integration of knowledge management and computer vision technologies for detecting and segmenting objects, estimating the three-dimensional human skeleton, tracking objects in video, estimating the ground plane normal to calculate the camera distance, and classification of short-term actions of people using three-dimensional skeleton. This method can be used in the development of a decision support system by operators of video monitoring systems used to detect and handle deviant behavior of people in real time in order to prevent escalation, provide timely aid to victims and detain suspects.