An Efficient Multi-Scale Attention two-stream inflated 3D ConvNet network for cattle behavior recognition

Abstract

Accurately identifying the basic motion behaviors of cattle, such as grazing and drinking, is crucial for monitoring their health status. Traditional manual monitoring methods are not only time-consuming and inefficient but also highly subjective, making continuous 24-hour surveillance challenging. Moreover, wearing physical sensors for extended periods can interfere with normal cattle activities, causing discomfort to the animals. Existing algorithms using video surveillance for detecting the basic motion behaviors of cattle have several shortcomings, including low model accuracy, poor robustness, and difficulties in effective real-world application. To overcome these shortcomings, this paper proposes a novel two-stream inflated EMAInception3D ConvNet (referred to as two-stream M3D), which consists of two parallel branches. The upper branch is the RGB M3D network, which processes the original RGB video frame sequence and extracts spatial features related to visual appearance. The lower branch is the Optical Flow M3D network, which processes the optical flow image frame sequence generated by calculating the differences between superimposed video frames. By learning from the optical flow images, the Optical Flow M3D network is able to capture temporal variations that are not discernible in static images, understand the correlations between successive action changes, and extract more in-depth motion features of cow actions in the temporal dimension. Finally, the outputs of the two branches are fused to extract richer and more robust features. Traditional single-scale feature extraction methods often overlook subtle multi-scale features. Therefore, we have introduced the Efficient Multi-Scale Attention Module (EMA) to enhance the network’s ability to capture details and filter background. Furthermore, to further improve the model’s capability in analyzing temporal dimensions and capturing long-term dependencies in behavior, we have incorporated a Non-Local module. The Non-Local module, by calculating the relationships between different positions in video sequences, enhances the network’s understanding of dynamic information. The two-stream M3D model, integrating EMA and Non-Local, can effectively utilize the spatio-temporal information of behavior videos to identify and analyze subtle changes in the basic motion behaviors of cattle. Compared with traditional methods, the model proposed in this study has achieved state-of-the-art recognition performance, and the accuracy of motion recognition was 94.281%, which was 1.771% higher than the two-stream I3D model.